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MESSAGE FROM THE DIRECTOR 


On behalf of the Student Evaluation and 
Records Branch, I am pleased to present the 
Achievement Testing Program Provincial 
Report. 


In June 1989, achievement tests were ad- 
ministered in Grade 3 English Language 
Arts, Grade 6 Social Studies, and Grade 9 
Science. Results show that overall student 
performance remains satisfactory. More 
students than expected in Grade 3 Language 
Arts achieved the acceptable standard but 
fewer students than expected in Grade 6 
Social Studies and in Grade 9 Science 
achieved the acceptable standard. The 
number of students achieving excellence 
exceeded expectations in all three testing 
areas. This provincial pattern of results is 
consistent with previous test administrations. 
I particularly draw readers’ attention to 
Section 7, where 1989 results are compared 
with those obtained in 1985. 


This year, more Francophone and French 
Immersion students than ever before 
participated in our testing program. For the 
first time, we are providing jurisdictions with 
their French language program results in 
relation to provincial data in order to open 
discussions on the most useful way of re- 
porting Francophone and French Immersion 
results. We also carried out special studies 
to find out the effects of language of testing 
on measuring achievement in the French 
Immersion Program. Results for French 
language testing will be available to admin- 
istrators and teachers in participating 
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jurisdictions and schools in the form of a 
special report. Other interested individuals 
may receive a copy of this special report upon 
request. 


Achievement test results provide educators 
with an opportunity to inform parents about the 
strengths of local programs and about initia- 
tives taken to address any weaknesses. We 
have included in Appendix F the answers to a 
number of questions frequently posed by par- 
ents concerning the Achievement Testing 
Program. Local boards, schools, and teachers 
may find it useful to reproduce and distribute 
these pages to interested parents or to use them 
as an item in district and school newsletters. 


The staff at the Student Evaluation and Records 
Branch have tried to make the Provincial 
Report more readable and informative. We 
hope you like these improvements. Please let 
us know your opinion by completing the ques- 
tionnaire at the end of this document. 


Finally, I wish to express special appreciation 
to those teachers, principals, and superinten- 
dents who shared their experience and expertise 
with the Student Evaluation and Records 
Branch during the rigorous achievement test 
development and marking processes. Your 
dedication to those tasks and to making certain 
that test conditions were uniform has once 
again enabled us to assess student achievement 
fairly and accurately across Alberta. We are 
pleased to have your assistance in providing 
valuable information about student achieve- 
ment to policymakers, educators, and the public. 


Lh 


Frank G. Horvath, Director 
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SECTION 1 


SUMMARY OF ACHIEVEMENT TEST RESULTS 


STUDENT POPULATION 

In June 1989, 91 286 achievement tests were 
administered to students in the province of 
Alberta. 


Figure 1-1 


Figure 1-1 shows the percentage of 
students in grades 3, 6, and 9 who wrote 
the achievement tests as well as those who 
were absent and exempt. 
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* Students in the English language program who were required to participate in the Achievement 


Testing Program. 
** See tables 3-3, 4-4, and 5-3. 


The number of students who were absent on 
the day the tests were administered or who 
were exempt from writing is shown in tables 
3-3, 4-4, and 5-3. 


RESULTS IN RELATION TO 
STANDARDS 


Through discussions with educators and 
school administrators and from our exper- 
ience with measuring student achievement 
according to the expectations in the Program 
of Studies, we believe that it is reasonable to 
expect that at least 85% of students should 
achieve at the acceptable level and at least 
15% of students should achieve at the level 
of excellence. As in previous years, results 


are reported in relation to these standards. 
Standards are based only on results 
achieved by English language program 
students who wrote the 1989 achievement 
tests and not on the total population. 
Results in relation to the acceptable 
standard are presented for the total 
population in Appendix G, page 69. The 
total population comprises all English 
language program students who wrote the 
test and those students who were absent or 
who were exempt from writing the test. 


Figures 1-2 and 1-3, page 2, present the 
percentage of students who met the 
acceptable standard and the standard of 
excellence. 


a 


Figure 1-2 


Percentage of Students Achieving Acceptable Standard 
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Results presented in figures 1-2 and 1-3 grades 6 and 9. For the standard of 
reveal that the percentage of students excellence, percentages were slightly 
achieving the acceptable standard was higher than were expected for grades 3 
slightly higher than was expected for and 6 and were considerably higher than 
Grade 3 but the respective percentages were expected for Grade 9. 


were lower than were expected for 
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SECTION 2 


GUIDELINES FOR INTERPRETING 
ACHIEVEMENT TEST RESULTS 


Following each administration of the 
achievement tests, a Provincial Report is 
prepared. This report is a public document 
that describes the aggregated results obtained 
by those students who wrote achievement 
tests in a given year. Provincial reports can 
be used by school board members, super- 
intendents, principals, and teachers as they 
review their own confidential jurisdiction 
and school reports. 


By using the Provincial Report in this way, 
policymakers and educators can check their 
perceptions of local achievement against 
provincewide standards and trends in the 
levels of achievement. 


This Provincial Report describes the results 
achieved by students who wrote the June 
1989 achievement tests in Grade 3 English 
Language Arts, Grade 6 Social Studies, and 
Grade 9 Science. 


The achievement test development process is 
described in Appendix B, page 59. 


Provincial results for students who wrote 
French translations of the Grade 6 Social 
Studies Achievement Test and the Grade 9 
Science Achievement Test will be presented 
in a special report. This report will be 
available to participating jurisdictions and 
schools. Other interested individuals may 
receive a copy of this special report upon 
request. 


JURISDICTION AND SCHOOL REPORTS 


In addition to the Provincial Report, 
superintendents and principals receive 
confidential reports of results achieved by 
students in their particular jurisdiction 


or school. The jurisdiction and school 
reports contain tables that parallel the 
major tables in the Provincial Report. 


Policymakers and educators in each juris- 
diction are encouraged to study carefully 
the provincial results and their own test 
results. 


Educators at the school and jurisdiction 
level can make two kinds of comparisons 
of the achievement of their students. One 
comparison is in relation to expectations or 
standards; the other is in relation to the 
achievement of students in the entire prov- 
ince of Alberta. 


As a result of these comparisons, teachers, 
principals, and superintendents can reflect 
on the programs that were delivered in 
their grades 3, 6, and 9 classrooms and 
make changes wherever necessary or 
desirable. 


USE OF THE JURISDICTION AND 
SCHOOL REPORTS 


The reports are NOT intended to be used 
as the basis for 


*making decisions about student place- 
ment or promotion 

*evaluating teacher performance 

*comparing performance between or 
among schools. 


Administrators in each jurisdiction should 
apply separate locally developed teacher, 
school, and school system evaluation 
policies to the tasks of evaluating teacher 
and school performance. 


The information provided in the reports is 
factual regarding what has happened as a 
result of the administration of the tests. The 
interpretation of this information -- 
hypothesizing why results are as they are -- 
involves consideration of the many factors 
and variables that contribute to achievement. 


In addition, it must be noted that the in- 
formation in these reports is itself limited to 
selected objectives of the Program of 
Studies. Many important aspects of learning 
cannot be measured by the achievement 
tests, which are time-limited paper and 
pencil tests. 


STANDARD SETTING 


Standards have been set for each of the 
achievement tests (see Appendix C, page 
63). Our judgment is that 85% of students 
should be able to meet or exceed the 
acceptable standard of achievement, and 
15% should be able to meet or exceed the 
standard of excellence. Included in the 
jurisdiction and school reports is a table 
showing the percentage of students meeting 
each standard set. The table also indicates 
whether the number of students in that 
school or jurisdiction who have achieved the 
standard is significantly different from the 
expected number (based on the 85% and 
15% expectations), and unlikely to be due to 
chance variation. 


For the purposes of the Achievement Testing 
Program, the 95% confidence interval is 
used. That is, if the probability is less than 
one in 20 that a difference is due to chance, 


this difference is very likely a real 
difference. 


Although the statistical tests take the 
number of students into consideration, it is 
a useful rule of thumb that results for 
groups of fewer that 25 students must be 
interpreted with particular caution. 

Chance variation in small groups is greater. 


Educators interpreting these reports are 
encouraged to consider how well their 
students have done compared to the 
standards. 


COMPARING RESULTS TO AVERAGE 
SCORES 


While overall test results are presented in 
relation to provincial standards, each 
jurisdiction and school report also 
provides jurisdiction or school average 
scores for each reporting category or 
subtest. Each of these scores may be 
compared to the provincial average for the 
same reporting category or subtest to 
determine if differences exist. 


The importance of differences that may 
exist between jurisdiction or school aver- 
ages and provincial averages is not always 
clear. To aid in the interpretation of 
differences between the averages, 
jurisdiction and school reports indicate 
when a difference is unlikely to be due to 
chance variation. The 95% confidence 
interval is also used here to identify signif- 
icant differences. 


FACTORS LIMITING THE INTERPRETATION OF TEST RESULTS 


Educators who are interpreting results must 
take into account the following limitations: 


1. Paper and pencil tests necessarily 
measure reading ability. Achievement 
tests are designed to have a readability 
level equivalent to the grade level being 
tested. Jurisdictions should consider the 
average reading level of their grades 6 
and 9 students, as reading levels below 
these grades will have an effect on test 
results that will be independent of 


achievement in social studies and 
science respectively. 


If more than 10% of eligible students 
in a jurisdiction did not write a test, 
the reported averages for that juris- 
diction may not accurately represent 
the true averages. 


Consideration should be given to the 
degree to which students in particular 
classes or grades were motivated to 
perform to their levels of ability. 


FACTORS THAT MAY AFFECT STUDENT ACHIEVEMENT 


There are many factors or variables that 
may effect student achievement. Some of 
these factors are: 


1. Environment 


* community environment 

¢ school environment 

* socioeconomic background 
¢ family circumstances 


2. Student Factors 


¢ ability 

* attitude 

* motivation 

° aspiration 

e academic background 
* learning style 


Resources (availability and 
appropriateness) 


* programs of study 

* curriculum of study 
* resource materials 

¢ library services 

* current textbooks 

* references 


Instruction 


* teacher qualifications 

* teacher experience 

¢ professional development 

* teacher morale 

* teaching strategies 

* hours of instruction 

¢ staff turnover 

* amount of homework assigned 

* communication of teacher 
expectations 


SECTION 3 


GRADE 3 ENGLISH LANGUAGE ARTS 


GENERAL DESCRIPTION 


The Grade 3 English Language Arts Achieve- 


ment Test was a two-part test. Part A: 
Writing was a 60-minute writing test con- 
sisting of a story starter and instructions for 
the student to finish writing the story. This 
format was designed to reflect the writing 
process. Part B: Reading was a 50-minute 
reading test consisting of 40 multiple-choice 
questions based on reading selections from 
fiction, nonfiction, and poetry. 


The test was designed to reflect the Grade 3 
Language Arts curriculum specifications that 
have been developed from the Program of 
Studies for Elementary School 1978 (amen- 
ded 1982). The scope of the Grade 3 English 
Language Arts Achievement Test was 
limited to the writing and the reading 
components of the program. 


The information presented in this section is 
based on the results achieved by 31 998 
students. 


SUMMARY OF RESULTS 
R ts in Relation to Standar 


Results show that 86.2% of students who 
wrote the test achieved the acceptable 
standard and 16.1% achieved the standard of 
excellence. These results were slightly 
higher than expectations for both standards. 
The acceptable standard and the standard of 
excellence were established by a standard- 
setting procedure (see Appendix C, page 63). 


Average Score 


The average total score for the test was 
68.9%, with a standard deviation of 15.7. 
The average raw score for Part A: Writing 
was 16.2 marks out of a possible 25, with a 


standard deviation of 4.1. For Part B: 
Reading, the average raw score was 29.1 
marks out of a possible 40, with a standard 
deviation of 8.1. 


CONTENT OF PART A: WRITING 


The Part A: Writing booklet included one 
page labelled IDEAS and several pages for 
completing the story. The writing assign- 
ment followed a story starter that was read 
by the student. The assignment set a 
specific writing task but allowed the student 
to use imagination and background exper- 
ience to develop a story. Papers were 
scored for Content, Development, Sentence 
Structure, Vocabulary, and Conventions. 


CONTENT OF PART B: READING 


Every effort was made to select complete 
passages for Part B: Reading. As well, 
reading selections were chosen to reflect 
the interests of the majority of Grade 3 
students and to be of appropriate difficulty 
for Grade 3 students. Extensive use was 
made of Canadian material. 


Questions were developed to test how well 
the students could understand and analyse 
the reading selections, and could make 
judgments about form and content. Only 
questions dealing with significant aspects of 
the reading selections were used. 


ACHIEVEMENT TEST BLUEPRINT 


Questions were classified according to two 
cognitive levels: Literal Understanding (12 
questions), and Inferential Understanding 
and Judgment (28 questions). By consid- 
ering cognitive level when developing 

a test, the Student Evaluation and Records 
Branch ensures that students will use a 
variety of mental activities as they write the 
test. 


Questions listed under Literal Understanding 
are designed to test the skills of recall and 
recognition; those listed under Inferential 
Understanding and Judgment are designed to 
test the skills of analysis, interpretation, 
extrapolation, and judgment. 


Table 3-1 presents the blueprint used to 
develop Part B: Reading. Classification by 
reporting category for each question in- 
cluded in Part B: Reading is indicated in the 
table. 


Table 3-1 
Grade 3 English Language Arts 
Part B: Reading 
Achievement Test Blueprint 


Reporting 
Category 


1. Attending to Details 
The student should be able 
to construct meaning from 
background experience 
and by attending to the 
supporting details found 
in a reading selection. 


. Associating Meanings 
The student should be able 


to associate meanings of 
words and expressions from 
background experience 
and from contextual clues 
in a reading selection. 


. Synthesizing Ideas 
The student should be able 
to synthesize ideas from 
the entire reading selection 
in order to construct 
meaning, to deduce the 
main idea, and to predict 
plausible outcomes or 
conclusions. 


Understanding 


Cognitive Level 


‘al Total 
Inferenti Nuniber 


Understanding of 
and 
Judgment 


2,9,14,17,18, 


23,25,27,28, 
32,3337 


Literal 


Questions 


1,5,13,16, 
19,24,26, 
29,30,31 


4,7,8,10,22, 
40 


3,6,15,20,21, 
34,35,36,38, 
39 


STUDENTS TESTED, ABSENT, AND EXEMPT 


Table 3-2 presents the number of students in this Provincial Report but will be 

who wrote the Grade 3 English Language provided in a special report, which will be 
Arts Achievement Test. Students in French available to administrators and teachers in 
Immersion or Francophone programs could participating jurisdictions and schools. 

be exempt from the test at the option of Another 160 students in French Immersion 
the superintendent. As the table shows, 2 092 or Francophone programs did not write the 
of these students wrote the test. Their scores test, as Table 3-3 indicates. 


are NOT included in the results given 


Table 3-2 
Grade 3 English Language Arts 
Students Tested 


T f Participati 
Participation Required 31 998« 
(Students in Regular Program) 


Participation Optional 
(Students in Francophone/French Immersion Programs) 


* Of the total number of students required to write the test, 1 146 students were absent the day the test was written 
and 1 500 students were exempt from writing the test. (See Table 3-3.) 
** Results achieved by these students are not included in the provincial data because participation in the Achieve- 
ment Testing Program is optional for these students. 


Table 3-3 presents the number and percent- Language Arts Achievement Test and who 
age distribution of students who were re- were absent or exempt. 
quired to write the Grade 3 English 


Table 3-3 
Grade 3 English Language Arts 
Students Included in Provincial Results, Absentees, and Exemptions 


Students of Students 


Students Included in Provincial Results 


Students Absent 


Students Exempt: 


RESULTS FOR THE TOTAL TEST 


To calculate a total test score for students, achieving the acceptable standard and the 

Part A: Writing and Part B: Reading were standard of excellence for the total test, for 

given equal weighting. A summary score for Part A: Writing, and for Part B: Reading. 

Part A: Writing was calculated by adding up 

the scores for each of the five written- The acceptable standard and the standard 

response scales, thus giving each scale equal of excellence were established by a 

weighting in the summary score. standard-setting procedure (see Appen- 
dix C, page 63). 

RESULTS IN RELATION TO 

STANDARDS 


Table 3-4 shows the percentage of students 


Table 3-4 
Grade 3 English Language Arts 
Percentage of Students Achieving Standards 


Reporting Category Score Percentage of Students 
and Representing Achieving At or 
Level of Standard Standard Above Standard 
Expected Actual 


Total Test (Maximum Possible Score = 100) 
Acceptable Standard 
Standard of Excellence 


Part A: Writing (Maximum Possible Raw Score = 25) 
Acceptable Standard 
Standard of Excellence 


Part B: Reading (Maximum Possible Raw Score = 40) 
Acceptable Standard 
Standard of Excellence 


-10- 


The numbers of students achieving the 
acceptable standard and the standard of 
excellence for each jurisdiction were 
analysed to determine whether jurisdictions 
were below expectations, meeting expec- 
tations, or above expectations. Jurisdictions 
classified as meeting expectations were those 
for which the difference between the actual 
number of students and the expected number 
of students at or above expectations was not 


statistically significant. A 95% confidence 
interval was used; this criterion means that 
differences are only reported when there is 
a 5% or smaller probability that a differ- 
ence of that size could occur by chance. 


The results are reported in Table 3-5. The 
percentages in the table are based on 207 
jurisdictions (including private schools). 


Table 3-5 
Grade 3 English Language Arts 
Percentage Distribution of Jurisdictions* Meeting Expectations 


Reporting Category Below 


Meeting Above 


and Expectations Expectations Expectations 


Level of Standard 


Total Test 
Acceptable Standard** 
Standard of Excellence*** 


Part A: Writing 
Acceptable Standard 
Standard of Excellence 


Part B: Reading 
Acceptable Standard 
Standard of Excellence 


* Jurisdictions with fewer than five students are excluded, as the statistical significance of the frequencies 


compared to the expectations cannot be calculated. 


** Acceptable Standard: 85% of students are expected to achieve at or above the acceptable standard. 
*« Standard of Excellence: 15% of students are expected to achieve at or above the standard of excellence. 


AVERAGE SCORE 


Another way to look at the achievement of 
students is by means of the average score. 


The average total score for the Grade 3 
English Language Arts Achievement Test 
was 68.9%, with a standard deviation of 
15.7. 


Se 


Table 3-6 shows the percentage of students who scored at or below each total test 


who obtained each total test score (relative score (cumulative frequency). Total test 
frequency) and the percentage of students scores are expressed as percentages. 
Table 3-6 


Grade 3 English Language Arts 
Frequency Distribution of Total Test Scores 


Total Relative Cumulative Relative Cumulative 
Score Frequency Frequency Frequency Frequency 
(%) (%) (%) (%) (%) 
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RESULTS FOR PART A: WRITING 


Raw scores were calculated by adding the 
marks earned for each of the five 5-point 
reporting categories. 


RESULTS IN RELATION TO 
STANDARDS 


The acceptable standard and the standard 
of excellence were established by a 
standard-setting procedure (see Appendix 
C, page 63). For Part A: Writing, the 
standard established was such that in order 
to meet 


the acceptable standard, students had 
to achieve a raw score of 13 out of 25 


*the standard of excellence, students 
had to achieve a raw score of 20 out 
of 25. 


Based on these standards, the results 
revealed that 


*84.5% of students performed at or 
above the acceptable standard, and 


°20.6% of students performed at or 
above the standard of excellence. 


These levels of performance were as high as 
were expected at the acceptable standard and 
were higher than were expected at the 
standard of excellence. 


AVERAGE SCORE 


The average raw score for Part A: Writing 
was 16.2 marks out of a possible 25, with a 
standard deviation of 4.1. 


Table 3-7 shows the percentage of students 
who obtained each score on Part A: Writing 
(relative frequency) and the percentage of 
students who scored at or below each score 
(cumulative frequency). 


Results for Part A: Writing are most clearly 
understood in the context of the assignment 
students responded to and in the context of 
the scoring descriptors. Complete scoring 
guides are available from the Student Eval- 
uation and Records Branch (427-2948). 


All schools should have extra copies of the 
Part A: Writing test to use in conjunction 
with information provided in this Provincial 
Report. 


Table 3-7 


Grade 3 English Language Arts 
Part A: Writing 
Frequency Distribution of Raw Scores 


Relative Cumulative 
Frequency Frequency 


(%) (%) 


0 
1 
2 
3 
4 
5 
6 
7 
8 
9 


Total Relative Cumulative 
Raw Frequency Frequency 
Score (%) (%) 


= 132 


SCORING RELIABILITY The results outlined in Table 3-8 are 
best considered in terms of the percent- 


Although the papers were scored on a age of students that markers judged to have 
one-marker system, 244 papers were presented work that was 3 (Satisfactory) 
re-marked so that a second set of scores or better for any reporting category. 
was available for these papers to confirm 
scoring consistency. Of the scores It is possible to draw conclusions about local 
awarded on the second reading, 90.8% program strengths and weaknesses by com- 
were identical to the original score on the paring local percentages of 3 (Satisfactory) 
same scale or varied by only one point. It or better scores on each reporting category 
is important to note that the one-marker with the provincial averages. 
system produces results that are reliable 
for groups of 25 or more students. Students do better on some dimensions of 
Achievement test scores are not intended the task than on others. (See Examiners’ 
to be reliable for individual students. Remarks, page 18.) 
Table 3-8 
Grade 3 English Language Arts 
Part A: Writing 


Percentage Distribution of Scores 


Score Reporting Category 
(Scale Points) Content Development Sentence Vocabulary Conventions 
Structure 


INS (insufficient or 
No Response) 


RESULTS FOR PART B: READING 


Since over 94.5% of students writing com- the acceptable standard, students had to 

pleted Part B: Reading, it was concluded that achieve a raw score of 21 out of 40 

sufficient time was allotted for writing the 

test. *the standard of excellence, students had to 
achieve a raw score of 36 out of 40. 

RESULTS IN RELATION TO 

STANDARDS Based on these standards, the results re- 

vealed that 

The acceptable standard and the standard of 

excellence were established by a standard- °82.7% of students performed at or above 

setting procedure (see Appendix C, page the acceptable standard, and 

63). For Part B: Reading, the standard 

established was such that in order to meet °25.4% of students performed at or above 


the standard of excellence. 
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The level of performance was slightly lower 
than was expected at the acceptable standard 
and was much higher than was expected at 
the standard of excellence. 


AVERAGE SCORE 


Provincial summary results for Part B: 
Reading were as follows: 


°Provincial Average -- 29.1 marks out of 
a possible 40 
*Standard Deviation -- 8.1 


As outlined in the blueprint on page 8, the 
questions on Part B: Reading were grouped 
according to reporting categories. 


Raw score averages for each of these re- 
porting categories and for Part B: Reading as 


a whole are presented in Table 3-9. Raw 
score averages were computed and 
rounded to one decimal. 


Although levels of performance in the dif- 
ferent reporting categories appeared to 
show some variation, caution is advised 
when comparing them. The sets of 
questions that made up each category were 
not selected to be equal in average level of 
difficulty; therefore, differences may have 
been due to variations in question diffi- 
culty rather than in student performance. 
The raw score averages can be used, 
however, in combination with jurisdiction 
and school results to detect patterns of 
relative strength or weakness in achieve- 
ment in each of the categories. 


Table 3-9 
Grade 3 English Language Arts 
Part B: Reading 
Raw Score Results by Reporting Category 


Reporting Number of 
Category Questions 


Raw Score Standard 
Average Deviation 


Total Part B: Reading 40 29.1 8.1 


Attending to Details 


Associating Meanings 


Synthesizing Ideas 
Literal Understanding 


Inferential Understanding 
and Judgment 
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Table 3-10 presents the percentage of stu- percentage of students who scored at or 
dents who obtained each score on Part B: below each score (cumulative frequency). 
Reading (relative frequency) and the 


Table 3-10 
Grade 3 English Language Arts 
Part B: Reading 
Frequency Distribution of Raw Scores 


Total Relative Cumulative Total Relative Cumulative 
Raw Frequency Frequency Raw Frequency Frequency 
Score (%) Score (%) (%) 


OonNAMN PWN KE © 
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PERCENTAGE OF STUDENTS CHOOSING EACH ALTERNATIVE 


Table 3-11 presents the percentage of students The results shown in Table 3-11 can best be 
who chose each alternative (A, B, C, and D) used in conjunction with results presented in 
for each multiple-choice question on Part B: jurisdiction and school reports in order to 
Reading. The correct response (key) for each interpret strengths and weaknesses of local 
question is also identified. programs. 
Table 3-11 
Grade 3 English Language Arts 
Part B: Reading 


Results for Individual Multiple-Choice Questions* 


Distribution of Distribution of 
Responses (%) Responses (%) 


*The sum of the percentages for each question may be less than 100% because the No Response category is not 
included. The No Response category does not exceed 5.4% for any one of these questions. 
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GRADE 3 ENGLISH LANGUAGE ARTS 
EXAMINERS’ REMARKS 


Teacher-markers and standard-setters felt that 
the Grade 3 English Language Arts Achieve- 
ment Test reflected the specifications of the 
curriculum, with one caution: teachers, during 
instruction, look at learning in a more holistic 
and developmental way. Every effort was 
made to address this perspective in developing 
the writing and reading components of this 
test. This test is a valid measure of student 
achievement in these two areas. 


PART A: WRITING 


Markers were very pleased with the overall 
quality of students’ writing. Students han- 
dled the narrative form very well; they created 
stories that reflected personal experience and 


ideas from literature and the media. They took 


risks with such elements as vocabulary, details 


’ 


sentence structure, and dialogue. Even most of 


the weaker students expressed ideas quite 
clearly -- which supports teacher comments 
that there is more writing going on in 
classrooms. 


The percentage of students who scored 


3 (Satisfactory) or better in 1989 was compared 


with the 1985 figures: 


1989 1985 
Content 82.3% 77.59% 
Development 77.1% 71.3% 
Sentence Structure 82.9% 77.7% 
Vocabulary 86.0% 79.5% 
Conventions 17.5% 78.3% 


Scores in 1989 increased on all marking scales 
except for Conventions. Only 0.2% of stu- 
dents, compared to 0.4% in 1985, produced 
written work that was considered to be 
Insufficient for scoring purposes. 


In summary, at least 77.1% of students scored 


3 (Satisfactory) or better on any marking scale. 


The highest achievement was on Vocabulary 
and the lowest was on Development. 


A booklet to provide Grade 3 teachers, ad- 
ministrators, and students with samples of 
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students’ writing that exemplify the criteria 
used to score written responses on the June 
1989 Grade 3 English Language Arts 
Achievement Test will be published in the 
near future and mailed to teachers. 


PART B: READING 


Teacher-markers and standard-setters felt 
that the reading section represented an 
appropriate range of difficulty for Grade 3 
students. They appreciated the variety of 
themes, the quality literature, and the 
Canadian content. 


Detailed statistical review of all questions 
revealed that students scored higher on 
questions that required Literal Understanding 
-- 77.5% -- than on questions that required 
Inferential Understanding and Judgment -- 
70.7%. 


Questions ranged from those that students 
found to be difficult, such as question 38 
with only 20.6% of students answering cor- 
rectly, to those that were easy for students, 
such as question 4 with 91.6% of students 
answering correctly. 


Question 38 proved to be difficult because it 
required a high level of thinking. The 
question read: 


38. Which sentence BEST tells us that the 
Rabbit was no longer a toy? 
°”*You will be REAL!”” 
"Love stirred in his little sawdust 
heart." 
"The Rabbit was wet through with 
the dew." 
The Boy and his Rabbit had long 
days in the garden.” 


It required students to recognize the signif- 
icance of the difference in tense between 

the stem and the first alternative. In addition, 
students had to be aware that love cannot be 
experienced by an inanimate object such as a 
toy. 


Question 4 asked students to infer the meaning 
of a word from context. The question read: 


4. 


In the story, what does the underlined word 
enormous mean? 

° White 

e Large 

¢ Lonely 

° Straight 
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Students found this to be a very easy task 
either because they are familiar with the 
word "enormous" or because they have a 
high level of skill in deriving meaning from 
context. 
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SECTION 4 


GRADE 6 SOCIAL STUDIES 


GENERAL DESCRIPTION 


The Grade 6 Social Studies Achievement 
Test was a two-part test. The time allotted 
for writing each part was 50 minutes. 


Part A: Multiple Choice consisted of 50 
questions worth 70% of the total test score. 


Part B: Written Response consisted of six 
short-answer questions and a composition 
question worth 30% of the total test score. 
All questions in this second part reflected a 
current social issue. 


The information presented in this section is 
based on the results achieved by 29 918 
students. 


SUMMARY OF RESULTS 
Results in Relation to Standards 


Results show that 81.6% of students who 
wrote the test achieved the acceptable 
standard and 16.6% achieved the standard 
of excellence. These results were lower than 
were expected for the acceptable standard 
and were slightly higher than were expected 
for the standard of excellence. The 
acceptable standard and the standard of 
excellence were established by a standard- 


setting procedure (see Appendix C, page 63). 


ver core 


The total test score was obtained by com- 
bining the scores for Part A: Multiple Choice 
and Part B: Written Response so that the two 
parts had a weighting of 70% and 30% 
respectively. 


The average total score for the test was 
62.5%, with a standard deviation of 16.3. 
The average raw score for Part A: Multiple 
Choice was 32.2 marks out of a possible 50, 


with a standard deviation of 8.9. For 

Part B: Written Response, the average raw 
score was 17.5 marks out of a possible 30, 
with a standard deviation of 5.1. 


CONTENT OF THE TEST 


The Grade 6 Social Studies Achievement 
Test was based on the 198] Alberta Social 
Studies Curriculum. All test questions 
were drawn from the content of the three 
topics prescribed for Grade 6: 


*Topic A: How People in Earlier Times 
Met Their Needs 


*Topic B: How People in Eastern Societ- 
ies Meet Their Needs Today 


*Topic C: Meeting Needs Through Local, 
Provincial, and Federal Governments 


Content emphases were drawn from the 
Grade 6 Social Studies Curriculum Spec- 
ifications. 


The Grade 6 Social Studies Achievement 
Test measured value, knowledge, and skill 
objectives. Objectives related to the de- 
velopment of attitudes and participation 
skills did not form part of this test. The 
weighting allocated to the development of 
these objectives in the curriculum specifi- 
cations was reassigned to the remaining 
objectives on a prorated basis. 


PART A: MULTIPLE CHOICE 
BLUEPRINT 


Table 4-1, page 22, presents the blueprint 
used to develop the Part A: Multiple 
Choice section of the test. Classification 
by reporting category for each question on 
the test is indicated in the table. 
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Table 4-1 
Grade 6 Social Studies 
Part A: Multiple Choice 
Achievement Test Blueprint 


Concept Reporting Category 


Value Concepts How People in Earlier Times | How People in Eastern Meeting Needs Through Local, 
Knowledge of competing Met Their Needs Societies Meet Their Needs Provincial, and Federal 
Process values or value positions Knowledge of facts, concepts, | Today Governments 
Reporting and generalizations related Knowledge of facts, concepts,| Knowledge of facts, concepts, 
Category to the meeting of needs in and generalizations related and generalizations related to 
earlier times to the meeting of needs in the meeting of needs through 
eastern societies governments 


5,6,7,8,9, 18,19,20,21, 43,44,45 46,4748, 
10,11 22,26,33 49,50 


Knowledge and 
Comprehension 


Recalls or recognizes 
data and transforms 
data into other words 


Inquiry Skills I 

Uses skills to identify 
an issue, select 
appropriate research 
questions, and gather 
and organize data 
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Inquiry Skills II 
Uses skills to analyse, 


evaluate, and synthesize 
data 


Inquiry Skills II] 
Uses skills to resolve an 


issue, apply a decision, 
and evaluate that decision 


Valuing Skill 424,31 ,4l 
Uses skills to analyse 
competing values 


Percent of Total Score 


PART B: WRITTEN-RESPONSE Question 7 asked students to write two or 
BLUEPRINT more paragraphs to persuade their 
classmates to adopt their position on an 
The written-response section of the test issue. The objectives on which the 
consisted of seven questions. Questions | written-response questions were based are 
to 6 required short-answer responses. shown in Table 4-2. 
Table 4-2 
Grade 6 Social Studies 


Part B: Written Response 
Achievement Test Blueprint 


Reporting Description of Proportion 
Category Writing Assignment of Total 
Score (%) 


I. Short Answer . Recalls facts related to an 
(Identification of the issue. Knowledge objectives -- 
Elements of an Issue) recall knowledge. 


. Recalls facts and applies them 
in a new situation. Skill 
objectives -- analyse and 
evaluate data. 


. Recalls facts and applies them 
in a new Situation. Skill 
objectives -- analyse and 
evaluate data. 


. Formulates a generalization. 
Skill objectives -- synthesize 
data. 


. Identifies speakers’ value 
positions. Value objectives -- 
develop an understanding of 
values and analyse values. 


. Identifies speakers’ value 
positions. Value objectives -- 
develop an understanding of 
values and analyse values. 


Subtotal 


IT. Composition 7. Presents and defends a 
(Resolution of an position. Skill objectives -- 
Issue) resolve the issue and 

communicate effectively. 


Subtotal 
Total 


STUDENTS TESTED, ABSENT, AND EXEMPT 


Table 4-3 presents the number of students 
who wrote the Grade 6 Social Studies 
Achievement Test or its French translation 
(6© Année Test de Rendement Etudes 
sociales). Students in French Immersion or 
Francophone programs could be exempt 
from the test at the option of the super- 
intendent. As the table shows, 77 of these 
students wrote the test in English, and 1 104 
wrote the test in French translation. Because 
their participation was optional, their scores 
are NOT included in the results given in this 
section of the Provincial Report. Results 
for students in French Immersion or Franco- 
phone programs who wrote the French 
translation will be presented in a special 
report, which will be available to 


administrators and teachers in partici- 
pating jurisdictions and schools. Of the 
173 students listed in Table 4-4 as exempt 
because the language of instruction was 
not English, an undetermined number 
were in French Immersion or Francophone 
programs. 


A special study to determine the effects of 
the language of testing on achievement 
scores was conducted by administering the 
Grade 6 Social Studies Achievement Test 
and its French translation to students in 
selected French Immersion Program 
classes. Results for students participating 
in this study will be provided in a special 
report under separate cover. 


Table 4-3 
Grade 6 Social Studies 
Students Tested 


Participation Required 
(Students Receiving Instruction in English) 


Participation Optional 

(Students Receiving Instruction in French) 
Wrote in English 

Wrote French Translation 


Selected Participation in Special Study 
(Students in French Immersion Program) 
Wrote in English 

Wrote French Translation 


are Number of 
T f Part t 
ype of Participation Srlenis 


29 918* 


77** 
1 104*## 


216*** 
22 1 *** 


* Of the total number of students required to write the test, 1 029 students were absent the day the test was written 
and 1 970 students were exempt from writing the test. (See Table 4-4.) 
** Results achieved by these students are not included in the provincial data because participation in the Achieve- 
ment Testing Program is optional for these students. 
*** Results achieved by these students will! be presented in a special report. 
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Table 4-4 presents the number and percent- Achievement Test and who were absent or 
age distribution of students who were re- exempt. 
quired to write the Grade 6 Social Studies 
Table 4-4 
Grade 6 Social Studies 
) Students Included in Provincial Results, Absentees, and Exemptions 
; 
) Number of Percentage 
rk Category Students of Students 


Students Included in Provincial Results 


) Students Absent 


] Students Exempt: 


RESULTS FOR THE TOTAL TEST 


RESULTS IN RELATION TO Part A: Multiple Choice, and for Part B: 
STANDARDS Written Response. 
Table 4-5 shows the percentage of students The acceptable standard and the standard 
achieving the acceptable standard and the of excellence were established by a 
standard of excellence for the total test, for standard-setting procedure (see Appen- 
dix C, page 63). 
Table 4-5 
Grade 6 Social Studies 


Percentage of Students Achieving Standards 


; Score Percentage of Students 
Reporting Cat 
ais es d vee Representing Achieving At or 
Level of Standard siancend ave Sandan 
xp u 


Total Test (Maximum Possible Score = 100) 
Acceptable Standard 
Standard of Excellence 


Part A: Multiple Choice (Maximum Possible Raw Score = 50) 
Acceptable Standard 
Standard of Excellence 


Part B: Written Response (Maximum Possible Raw Score = 30) 
Acceptable Standard 
Standard of Excellence 
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The numbers of students achieving the Statistically significant. A 95% confidence 


acceptable standard and the standard of interval was used; this criterion means that 
excellence for each jurisdiction were differences are only reported when there is 
analysed to determine whether jurisdictions a 5% or smaller probability that a differ- 
were below expectations, meeting expec- ence of that size could occur by chance. 
tations, or above expectations. Jurisdictions 

classified as meeting expectations were those The results are reported in Table 4-6. The 
for which the difference between the actual percentages in the table are based on 187 
number of students and the expected number jurisdictions (including private schools). 


of students at or above expectations was not 


Table 4-6 
Grade 6 Social Studies 
Percentage Distribution of Jurisdictions* Meeting Expectations 


Reporting Category Below Meeting Above 
and Expectations Expectations Expectations 
Level of Standard 


Total Test 
Acceptable Standard** 
Standard of Excellence*** 


Part A: Multiple Choice 
Acceptable Standard 
Standard of Excellence 


Part B: Written Response 
Acceptable Standard 
Standard of Excellence 


*Jurisdictions with fewer than five students are excluded, as the statistical significance of the frequencies 
compared to the expectations cannot be calculated. 
** Acceptable Standard: 85% of students are expected to achieve at or above the acceptable standard. 
*** Standard of Excellence: 15% of students are expected to achieve at or above the standard of excellence. 


AVERAGE SCORE . The average score for the total Grade 6 
Social Studies Achievement Test was 
Another way to look at the achievement of 62.5%, with a standard deviation of 16.3. 


students is by means of the average score. 
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Table 4-7 shows the percentage of students who scored at or below each total test 
who obtained each total test score (relative score (cumulative frequency). Total test 
frequency) and the percentage of students scores are expressed as percentages. 


Table 4-7 
Grade 6 Social Studies 
Frequency Distribution of Total Test Scores 


Total Relative Cumulative Relative Cumulative | 
Score Frequency Frequency Frequency Frequency 
(%) (%) (%) (%) (%) 
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RESULTS FOR PART A: MULTIPLE CHOICE 


RESULTS IN RELATION TO 
STANDARDS 


The acceptable standard and the standard of 
excellence were established by a standard- 
setting procedure (see Appendix C, page 

63). For Part A: Multiple Choice, the 
standard established was such that in order to 
meet 


ethe acceptable standard, students had to 
achieve a raw score of 23 out of 50 


the standard of excellence, students had to 
achieve a raw score of 40 out of 50. 


Based on these standards, the results revealed 
that 


°83.3% of students performed at or above 
the acceptable standard, and 


°24.3% of students performed at or above 
the standard of excellence. 


The level of performance was slightly lower 
than was expected at the acceptable standard 
and was much higher than was expected at 
the standard of excellence. 


AVERAGE SCORE 


Provincial summary results for Part A: 
Multiple Choice were as follows: 


*Provincial Average -- 32.2 marks out of 
a possible 50 
¢Standard Deviation -- 8.9 


As outlined in the blueprint on page 22, the 
questions on Part A: Multiple Choice were 
grouped according to reporting categories. 


Table 4-8, page 29, presents provincial 
averages for these reporting categories. 


Provincial averages were computed 

and rounded to one decimal. Consequently, 
the sum of the averages for the reporting 
categories is not the same as the average for 
the total test. 


Although levels of performance in the dif- 
ferent reporting categories appeared to show 
some variation, caution is advised when 
comparing them. The sets of questions that 
made up each category were not selected to 
be equal in average level of difficulty; 
therefore, differences may have been due to 
variations in question difficulty rather than 
in student performance. Jurisdiction and 
school results can be usefully compared 
with the provincial averages to detect 
patterns of relative strength or weakness in 
achievement in each of the reporting 
categories. 


- 28 - 


Table 4-8 shows the averages and standard categories specified by the blueprint for 


deviations for each of the reporting Part A: Multiple Choice. 
Table 4-8 
Grade 6 Social Studies 


Part A: Multiple Choice 
Raw Score Results by Reporting Category 


Reporting Number of Raw Score Standard 
Category Questions Average Deviation 


Topic A: How People in Earlier 
Times Met Their Needs 


Topic B: How People in Eastern 
Societies Meet Their Needs Today 


Topic C: Meeting Needs Through 
Local, Provincial, and Federal 
Governments 


Knowledge and Comprehension 
All Topics 


Topic A 
Topic B 
Topic C 
Value Concepts and Valuing 
Skills (All Topics) 
Inquiry Skills I 
(Ali Topics) 


Inquiry Skills II 
(All Topics) 


Inquiry Skills I 
(All Topics) 
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Table 4-9 shows the percentage of stu- percentage of students who scored at or | 
dents who obtained each score on Part A: below each score (cumulative frequency). 
Multiple Choice (relative frequency) and the | 


Table 4-9. | 
Grade 6 Social Studies 1 
Part A: Multiple Choice 
Frequency Distribution of Raw Scores 


Relative | Cumulative 
Raw Frequency Frequency 
(%) (%) 


Relative Cumulative 
Raw Frequency Frequency 
(%) 


230% 
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PERCENTAGE OF STUDENTS CHOOSING EACH ALTERNATIVE 


Table 4-10 presents the percentage of students The results shown in Table 4-10 can best be 
who chose each alternative (A, B, C, and D) used in conjunction with results presented in 
for each multiple-choice question on Part A: jurisdiction and school reports in order to 
Multiple Choice. The correct response (key) interpret strengths and weaknesses of local 
for each question is also identified. programs. 
Table 4-10 
Grade 6 Social Studies 


Part A: Multiple Choice 
Results for Individual Multiple-Choice Questions* 


Distribution of Distribution of 
Responses (%) Responses (%) 


*The sum of the percentages for each question may be less than 100% because the No Response category is not 
included. The No Response category does not exceed 2.7% for any one of these questions. 
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RESULTS FOR PART B: WRITTEN RESPONSE 


RESULTS IN RELATION TO STANDARDS 


The acceptable standard and the standard of 
excellence were established by a standard- 
setting procedure (see Appendix C, page 63). 
For Part B: Written Response, the standard 
established was such that in order to meet 


*the acceptable standard, students had to 
achieve a raw score of 15 out of 30 


*the standard of excellence, students had to 
achieve a raw score of 24 out of 30. 


Based on these standards, the results revealed 
that 


°72.9% of students performed at or above 


the acceptable standard, and 


°11.9% of students performed at or above 
the standard of excellence. 


Both levels of performance were much lower 
than were expected. See Examiners’ 
Remarks, page 34, for a comparison with 
1985. 


AVERAGE SCORE 

The average raw score for Part B: Written 
Response was 17.5 marks out of a possible 30, 
with a standard deviation of 5.1. 


The results for each written-response question 
are summarized in Table 4-11. 


Table 4-11 
Grade 6 Social Studies 
Part B: Written Response 
Average Scores Awarded 


; Total Marks 
Question Possible 


Short Answer 
1 


Composition 
7 


Average Difficulty 
Score Level* 


*The difficulty level is the average score divided by the total marks possible. 


sy Ae 


Table 4-12 presents the distribution of scores for the six short-answer questions. 


Table 4-12 
Grade 6 Social Studies 
Part B: Written Response 
Distribution of Scores for Short-answer Questions 


——E—E—E—=E—E—=_ re ss of Students — Each Mark 
24.3 27.8 47.8 


Table 4-13 presents the distribution of scores for the one composition question. 


Table 4-13 
Grade 6 Social Studies 
Part B: Written Response 
Distribution of Scores for Composition Question 


Persuasiveness Language and 
and Logic (%) Expression (%) 


2 (Limited) 
1 (Poor) 


0 (blank paper, off topic, insufficient 
response, illegible) 
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GRADE 6 SOCIAL STUDIES 
EXAMINERS’ REMARKS 


With the exception of the percentage of 
students achieving the acceptable standard 
on Part B: Written Response, which re- 
mained approximately the same as in 1985, 
more students achieved the acceptable 
standard and the standard of excellence in 
1989 than in 1985. (See Appendix C, page 
63.) Thus, while only 72.9% achieved the 
acceptable standard on Part B: Written 
Response, and 11.9% achieved the standard 
of excellence, this compares favorably with 
the results achieved in 1985. 


The Part A: Multiple Choice results were 
almost as high as were expected. At the 
provincial level, students appear to be 
mastering satisfactorily the knowledge, 
skills, and value objectives of the 
curriculum. The percentage of students 
achieving the standard of excellence on the 
multiple-choice part is particularly 
encouraging. 


The Part B: Written Response assignment 
required students to follow the steps of the 
inquiry process to address the issue of 
whether "the government should continue to 
help unemployed people meet their basic 
needs." This issue was chosen for its cur- 
ricular fit and topicality, and because 
students could legitimately take either side. 
As well, the topic permitted assessment of 
students’ ability to deal with abstract 
concepts. 


As in 1985, the majority of students appeared 
to experience difficulty in forming a general- 
ization. When asked in question 4 "What 
general statement can be made about meet- 
ing needs when a person is unemployed?", 
many students responded with advice for the 
unemployed rather than with a generalization 
that drew a relationship between unemploy- 
ment and meeting basic needs. 


Most students (89%) were able to identify 
the value positions underlying a speaker’s 


statement, and to identify which speakers 
held opposing positions. Fewer students 
(60%) were able to provide evidence from 
the speakers’ statements to demonstrate 
that the speakers held these values. Some 
students misunderstood the task required 
of them, and instead of providing evidence 
of the speakers’ values, offered their own 
opinion on the issue. 


The "evidence" portion of questions 5 and 
6 created difficulty for markers. Markers 
were instructed to distinguish between 
those papers that included some super- 
fluous editorializing and those in which 
extraneous material was sufficiently in- 
trusive that students failed to address the 
assigned task. Inter-rater reliability was 
lowest on these two items, which suggests 
that there was some variation in where 
individual markers drew this line. This 
seems to reflect a fundamental disagree- 
ment between those markers who felt that 
students who offered their own opinion 
were "going beyond the task required" but 
deserved marks for effort, and those who 
felt that these particular students had 
failed to address the assigned task and 
therefore should receive a zero. 


Papers that received a zero for question 7, 
the composition question, were reviewed. 
Of the 470 tests, 375 were blank for that 
question and two were illegible. The 
remaining 93 were off topic. 


Markers commented that some students 
lost marks because they confused the issue 
of unemployment with retirement. This 
may have been the result of students over- 
generalizing from practice exercises with 
the 1985 achievement test, which featured 
the issue of how best to care for the aged. 
Other students lost marks because they 
assumed the unemployed were disabled. 
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SECTION 5 


GRADE 9 SCIENCE 


GENERAL DESCRIPTION 


The Grade 9 Science Achievement Test 
consisted of 75 multiple-choice questions. 
The time allotted for writing the test was 90 
minutes. 


The information presented in this section is 
based on the results achieved by 27 137 
students. 


SUMMARY OF RESULTS 
Results in Relation to St 


Results show that 80.4% of students who 
wrote the test achieved the acceptable 
standard and 22.9% achieved the standard of 
excellence. These results were lower than 
were expected for the acceptable standard 
and were higher than were expected for the 
standard of excellence. The acceptable 
standard and the standard of excellence were 
established by a standard-setting procedure 
(see Appendix C, page 63). 


Average Score 


The average total test score was 66.8%, with 
a standard deviation of 17.7. The average 
total test raw score was 50.1 marks out of a 


possible 75, with a standard deviation of 13.3. 


CONTENT OF THE TEST 


The Grade 9 Science Achievement Test was 
designed to reflect the Grade 9 Science 
Curriculum Specifications (revised May 
1986). However, the scope of the test was 


limited to curriculum objectives that could 
be efficiently measured on a paper and 
pencil test. As a result, questions on the 
test were drawn from the content of the 
two major components in the core 
program: 


*Subject Matter 
Process Skills 


The subject matter component consisted of 
questions associated with the major con- 
cepts of the study of physical science in 
Grade 9. The process skills component 
consisted of questions integrated with 
subject matter and questions independent 
of subject matter. 


Questions on the Grade 9 Science Achieve- 
ment Test measured student achievement 
at three cognitive levels: 


*Knowledge -- recognize or recall 
ideas, terminology, facts, conventions, 
methods of inquiry, principles, gener- 
alizations, theories, and concepts 


*Comprehension and Application -- 
demonstrate an understanding of the 
concepts and skills, and apply approp- 
riate methods and ideas to a new 
situation 


*Higher Mental Activities -- demonstrate 
an ability to analyse and synthesize data 
in an effort to make generalizations, and 
evaluate ideas, solutions, and information. 
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Table 5-1 presents the blueprint used to Classification of each question by com- 


develop the Grade 9 Science Achievement ponent, subtest, and cognitive level is 
Test. indicated in the table. 
Table 5-1 
Grade 9 Science 


Achievement Test Blueprint 


Cognitive Levels* Component 
Reporting Number Number 
Category of Subject of 
Matter | Questions 


2,3,5,6,8, yer ey 1,4,5,6,8, 
11,12,13 F 10,12,13 


15,16,17, 21,22,24, | 14,15,16, 

19,20,21, 26,28,32, | 17,18,19, 

24,25,26, 20,23,25, 

27,30,31, 27,29,30, 
31 


36,37,38, 36,43,44, | 34,35,37, 
40,41,42, 45,50,51 | 38,39,40, 
43,45,47, 41,42,46, 

47 48,49 


68,69,70, 
713J2,73; 
74,75 


*K - - Knowledge 
C/A - - Comprehension and Application 
HMaA - - Higher Mental Activities 


= 36 


STUDENTS TESTED, ABSENT, AND EXEMPT 


The Grade 9 Science Achievement Test was of students who wrote the Grade 9 Science 
available both in English and in French Achievement Test or its French translation 
translation. Table 5-2 presents the number (9© Année Test de Rendement Sciences). 


Table 5-2 
Grade 9 Science 
Students Tested 


get ce Number of 
t 
Type of Participation cna 


27 137% 


Participation Required 
(Students Receiving Instruction in English) 


Participation Optional 
(Students Receiving Instruction in French) 
Wrote in English 

Wrote French Translation 


*Of the total number of students required to write the test, 1 205 students were absent the day the test was written 
and 1 318 students were exempt from writing the test. (See Table 5-3.) 
**Results achieved by these students are not included in the provincial data because participation in the Achieve- 
ment Testing Program is optional for these students. 
*** Results achieved by these students will be presented in a special report. 
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Table 5-3 presents the number and percent- Achievement Test and who were absent or 
age distribution of students who were exempt. 
required to write the Grade 9 Science 


Table 5-3 
Grade 9 Science 
Students Included in Provincial Results, Absentees, and Exemptions 


Category Number of Percentage 
Students of Students 
Students Included in Provincial Results 


Students Absent 


Students Exempt: 


RESULTS FOR THE TOTAL TEST 


RESULTS IN RELATION TO for the acceptable standard and were higher 
STANDARDS than were expected for the standard of 

excellence. The acceptable standard and the 
Table 5-4 shows the percentage of students standard of excellence were established by a 
achieving the acceptable standard and the standard-setting procedure (see Appendix C, 
standard of excellence. These levels of page 63). 


performance were lower than were expected 


Table 5-4 
Grade 9 Science 
Percentage of Students Achieving Standards 


Raw Score Percentage of Students 
Representing Achieving At or 
Standard* Above Standard 
Expected Actual 


Level of Standard 


Acceptable Standard 


Standard of Excellence 


*The maximum possible raw score was 75. 
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The numbers of students achieving the 
acceptable standard and the standard of 
excellence for each jurisdiction were 
analysed to determine whether jurisdictions 
were below expectations, meeting expec- 
tations, or above expectations. Jurisdictions 
classified as meeting expectations were those 
for which the difference between the actual 
number of students and the expected number 
of students at or above expectations was not 


statistically significant. A 95% confidence 
interval was used; this criterion means that 
differences are only reported when there is 
a 5% or smaller probability that a differ- 
ence of that size could occur by chance. 


The results are reported in Table 5-5. The 
percentages in the table are based on 173 
jurisdictions (including private schools). 


Table 5-5 
Grade 9 Science 
Percentage Distribution of Jurisdictions* Meeting Expectations 


Level of Standard Below 
Expectations 


Total Test 
Acceptable Standard** 
Standard of Excellence*** 


Meeting Above 
Expectations Expectations 


‘Jurisdictions with fewer than five students are excluded, as the statistical significance of the frequencies 


compared to the expectations cannot be calculated. 


**Acceptable Standard: 85% of students are expected to achieve at or above the acceptable standard. 
***Standard of Excellence: 15% of students are expected to achieve at or above the standard of excellence. 


AVERAGE SCORE 


Another way to look at the achievement of 
students is by means of the average score. 


The average score for the total Grade 9 
Science Achievement Test was 66.8%, 
with a standard deviation of 17.7. 
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REPORTING CATEGORIES 


Table 5-6 shows the total marks possible and the results shown in Table 5-6 can best be 
the provincial raw score results for the re- used in conjunction with parallel tables in 
porting categories of the Grade 9 Science the jurisdiction, school, and classroom 
Achievement Test. reports. Variations in patterns of students’ 

responses to questions can help to indicate 
It is important to stress that the averages on strengths and weaknesses in local educa- 
the various reporting categories cannot be tional programs. 


directly compared with one another. Rather, 


Table 5-6 
Grade 9 Science 
Raw Score Results by Reporting Categories 


Reporting Total Marks Raw Score Standard 
Category Possible Average Deviation 


Total Test 


Major Components 
Subject Matter 
Process Skills 


Subtests 

Matter Occupies Space 
Kinetic Molecular Theory 
Heat and Temperature 


Energy 


Atoms and Molecules 
Process Skills as Content 


Cognitive Levels 
Knowledge 
Comprehension/Application 
Higher Mental Activities 
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FREQUENCY DISTRIBUTION OF RAW SCORES 


Table 5-7 presents the percentage of students frequency) and the percentage. of students 
who obtained each score on the Grade 9 who scored at or below each score (cumu- 
Science Achievement Test (relative lative frequency). 

Table 5-7 


Grade 9 Science 
Frequency Distribution of Raw Scores 


Relative Cumulative Total Relative Cumulative 
Frequency Frequency Raw Frequency Frequency 
(%) Score (%) (%) 


0 
1 
2 
3 
4 
5 
6 
7 
8 
9 


PERCENTAGE OF STUDENTS CHOOSING EACH ALTERNATIVE 


Table 5-8 shows the percentage of students used in conjunction with the parallel tables 
who chose each alternative (A, B, C, and D) in the jurisdiction and school reports. 

for each multiple-choice question. The Variations in patterns of students’ re- 
correct response (key) for each question is sponses to questions can help to indicate 
also identified. strengths and weaknesses in local 


educational programs. 
The results shown in Table 5-8 can best be 


Table 5-8 
Grade 9 Science 
Results for Individual Multiple-Choice Questions* 


Distribution of Distribution of 


Responses (%) Responses (%) 
Item Key D B 


1 
2 
3 
4 
5 
6 
7 
8 
9 


included. The No Response category does not exceed 0.4 % for any one of these questions. 
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GRADE 9 SCIENCE 
EXAMINERS’ REMARKS 


Grade 9 Science teachers who were chosen 
as markers and standard-setters for the 1989 
Grade 9 Science Achievement Test felt that, 
generally, the test reflected the essence of the 
Grade 9 Science Program. In addition, 
teachers agreed that the test presented many 
questions that required students to think and 
to apply both science process and inquiry 
skills to analyse real-life situations. 
Standard-setting results in Table 7-6, page 
55, indicate that in 1989, 80.4% of students 
met the acceptable standard. Results indicate 
that slightly more students, 81.3%, met the 
acceptable standard in 1985. In essence, the 
number of students achieving the acceptable 
standard has not changed appreciably from 
1985 to 1989. 


In 1989, 22.9% of students met or exceeded 
the standard of excellence. However, com- 
parable standards set in 1989 for the 1985 
test indicate that 19.5% of students met or 
exceeded the standard of excellence. This 
suggests that a slightly higher number of 
students achieved at or above the standard of 
excellence in 1989 than in 1985. 


Teacher-markers felt that students were 
generally comfortable with the reading level 
of the questions. Students’ strengths and 
weaknesses in both subject matter and 
process skill questions are discussed below. 


Question 2 required students to identify the 
correct measurement for the line below a 
centimeter ruler. Many students (37.3%) 
chose 10.6 cm instead of 9.6 cm as the 
correct answer. This suggests that students 
failed to recognize that the line started at the 
1.0 cm mark. As a result, students may have 
read the length directly above the line on the 


ruler without subtracting 1.0 cm from 10.6 
cm. 


Question 7 required students to follow a 
four-step procedure to calculate the volume 
of a glass rod. A difficulty level of 51.9% 
indicates that some students experienced 
problems in sorting out relevant and irrel- 
evant data. 


Question 36 required students to identify 
the revision that would most likely improve 
an experiment. Most (80.4%) had little 
difficulty choosing the correct revision. 
This result seems to indicate that most 
Grade 9 Science students have little trouble 
analysing data and revising experiments of 
this type. 


Question 45 (difficulty level of 59.7%) re- 
quired students not only to interpret from a 
graph but also to identify the process that 
might take place if heat were removed from 
Material X at 160°C. The results for ques- 
tions 3 (difficulty level of 80.1%) and 27 
(difficulty level of 79.2%), which also con- 
cern graphs, suggest that interpreting 
graphs is a skill that most Grade 9 Science 
students possess. The difficulty they had 
with question 45, therefore, may be the 
inability of many students to integrate their 
knowledge and skills and transfer them to a 
new situation. 


When comparing the results of the 1989 
and 1985 Science achievement tests, it 
seems that, although students have shown 
some improvement in their ability to recall 
and/or recognize facts, ideas, concepts, and 
generalizations, overall student achieve- 
ment in 1989 appears to be unchanged 
compared to 1985. 
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SECTION 6 


STUDENT ACHIEVEMENT BY GENDER 


The figures and tables in this section of the The interpretation of this information -- 
report provide a breakdown by gender of the hypothesizing why results are as they are -- 
results achieved by students who wrote the requires thoughtful consideration of the 
June 1989 achievement tests. numerous variables that contribute to 
achievement. (See Section 2, page 3, and 
This information is a descriptive report of Appendix A, page 57, to interpret these 
male and female student achievement in results.) 
Grade 3 English Language Arts, Grade 6 
Social Studies, and Grade 9 Science. It is a Figure 6-1 shows the number of students by 
report of what happened in the June 1989 gender who wrote achievement tests in June 
Achievement Testing program. 1989. 


Figure 6-1 
Number of Tests Written 


(thousands) By Gender, for Each Subject 


Social St. (Gr. 6) Science (Gr. 9) 


fe Female 
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Figure 6-2 presents the percentage of students achieving the acceptable standard by gender. 


Figure 6-2 


Percentage of Students Achieving Acceptable Standard 


By Gender, for Each Subject 


Lang. Arts (Gr. 3) Social St. (Gr. 6) 


Male ee Female 


--- Expected Percentage of Students 


Science (Gr. 9) 


Figure 6-3 presents the percentage of students achieving the standard of excellence by gender. 


Figure 6-3 
Percentage of Students Achieving Standard of Excellence 


By Gender, for Each Subject 


Lang. Arts (Gr. 3) Social St. (Gr. 6) Science (Gr. 9) 


Male Be Female 


—-— Expected Percentage of Students 
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Table 6-1 presents the number and percentage of students by gender who wrote the June 1989 
achievement tests. 


Table 6-1 
Number and Percentage of Students Writing 
Achievement Tests by Gender 


Subject Number of Percentage 
Students of Students 


Grade 3 English Language Arts N=31 649* 
Male 
Female 


Grade 6 Social Studies N=29 788* 
Male 
Female 


Grade 9 Science N=27 008* 
Male 
Female 


*Gender was not reported for all students; therefore, totals differ slightly from those given elsewhere in this report. 
e 


Table 6-2 presents average scores and standard deviations by gender. 


Table 6-2 
Grades 3, 6, and 9 Achievement Tests 
Average and Standard Deviation by Gender 


Total Score Multiple Choice | Written Response 
(%) (Raw Score) (Raw Score) 
Male Female Male Female Male Female 


rade 3 English (Max Possible=40) | (Max Possible=25) 
Language Arts 
Average 28.5 29.7 15.7 16.8 
Standard Deviation 8.3 7.8 4.0 4.0 


Grade 6 Social Studies (Max Possible=50) | (Max Possible=30) 
Average 31.7 32.0 16.7 18.4 
Standard Deviation 9.1 8.9 Sel 5.0 


Grade 9 Science Grade 9 Science consists of 


Average multiple-choice questions only. 
Standard Deviation 
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SECTION 7 


STUDENT ACHIEVEMENT OVER TIME 


INTRODUCTION 


PURPOSE 


An important goal of Alberta Education is to 
measure and report changes in student 
achievement. Comparing student perform- 
ance on the 1985 and the 1989 achievement 
tests is one way of meeting this goal. A 
direct comparison of the average scores on 
the 1985 and the 1989 tests cannot be made; 
although the achievement tests were parallel 
in form and content, their levels of difficulty 
may not have been equal. A study was 
therefore undertaken to permit comparisons 
of student performance in 1985 and in 1989. 


STUDY DESIGN 


The study consisted of two parts for the 
Grade 3 English Language Arts and Grade 6 
Social Studies tests: test equating and 
standard setting. The Grade 9 Science study 
consisted of three parts: test equating, stan- 


dard setting, and comparing equivalent items. 


To equate the 1985 and the 1989 test scores, 
samples of grades 3, 6, and 9 students who 
completed the 1989 achievement tests also 
wrote the 1985 Grade 3 English Language 
Arts, Grade 6 Social Studies, and Grade 9 
Science achievement tests. The students 
selected for the study were from schools 
chosen to be representative of rural, urban 
large, and small schools throughout the 
province. 


? 


The 1985 and the 1989 multiple-choice 
scores from these students were used to 
establish what the 1985 equivalent scores 


would be on the 1989 test. (For an ex- 
planation of the multiple-choice test- 
equating process, refer to Appendix D, 
page 64.) Based on this equating process, 
the scores for the 1989 population were 
converted to estimated equivalent 1985 
scores, and the 1985 and the 1989 
populations were then compared. 


An additional procedure was required to 
equate the written-response sections of the 
1985 and the 1989 English Language Arts 
and Social Studies achievement tests (see 
Appendix E, page 65.) 


The standard-setting part of the study invol- 
ved groups of experienced teachers of 
grades 3, 6, and 9. Their task was to esta- 
blish the scores that represent the 
acceptable standard and the standard of 
excellence on each of the three achievement 
tests in 1985 and in 1989. The percentage 
of students meeting each standard for the 
respective tests was then compared. 


In addition to test equating and standard 
setting, a third procedure was used to 
compare student achievement in Grade 9 
Science. This procedure involved com- 
paring achievement on items in 1989 that 
were equivalent to those of 1985. Seven 
items from the 1985 test were paired with 
seven equivalent items from the 1989 test. 
The average score obtained by the 1985 and 
the 1989 populations of students on their 
respective sets of equivalent items was 
compared. 
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LIMITATIONS OF THE STUDY 


Many factors other than changes in levels of 
achievement could have contributed to dif- 
ferences in student performance on the 1985 
and the 1989 achievement tests. The 
Achievement Testing Program has been in 
place since 1982, and the 1985 tests were the 
first for those particular combinations of 
grade and subject. All those concerned have 
had more experience and had opportunity to 
adjust to the program since 1985. Content, 


understanding, and use of the bulletin 
provided by the Student Evaluation and 
Records Branch may have resulted in 
improved preparation for testing. Teachers 
in 1989 would have had an opportunity to 
use the 1985 tests for practice and to fam- 
iarize students with test-writing 
techniques. This opportunity did not exist 
in 1985. The effects of these extraneous 
variables could not be controlled; caution is 
therefore required when drawing conclu- 
sions about the results of the study. 


GRADE 3 ENGLISH LANGUAGE ARTS 


In 1989, the Grade 3 English Language Arts 
Achievement Test was administered to 

34 090 students enrolled in the Grade 3 
English Language Arts program. 


The 1985 and the 1989 Grade 3 English 
Language Arts achievement tests consisted 
of two parts, Part A: Writing (the 1985 
booklet was entitled Part A: Composition) 
and Part B: Reading. On the 1985 test, Part 
B: Reading contained 36 multiple-choice 
questions; on the 1989 test, there were 40 
multiple-choice questions. 


The average total test score in 1985 was 


66.9%; the average score in 1989 was 68.9%. 


The special comparison study involved 
administering the 1985 test to a sample of 
students who were also writing the 1989 test 
and using a single group of judges to deter- 
mine the standards for both tests. 


COMPARISON THROUGH 
RE-ADMINISTERING THE 1985 TEST 


In June 1989, 215 Grade 3 students from 
eight schools in eight jurisdictions were 
selected for test-equating purposes. (For an 
explanation of the test-equating process, 
refer to Appendix D, page 64, and Appen- 
dix E, page 65.) The schools were chosen to 
be representative of rural, urban, large, and 
small schools throughout the province. 
Based on the results of the test-equating 
process, the total scores for the 1989 popu- 
lation were converted to estimated equivalent 
1985 scores, and the 1985 and the 1989 
populations were then compared. Table 7-1 
shows the results of the comparison. 


Table 7-1 
Grade 3 English Language Arts 
Comparison of 1985 and 1989 Test Results 


Number of Marks 
Number of Students Writing 


Average Score after Equating 
Standard Deviation after Equating 
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The average 1989 score was higher than the 
average 1985 score by 1.6 marks out of a 
possible 100. A two-tailed t-test showed that 
this difference was statistically significant 
beyond the 0.001 level of probability: 
therefore, the difference is real and large 
enough to be of practical significance. 


STANDARD SETTING 


Twenty experienced Grade 3 English 
Language Arts teachers from schools 


throughout the province were selected to 
participate in the standard-setting section of 
the study. These teachers were among those 
marking the written-response part of the 
1989 test. (See Appendix C, page 63, for 
an explanation of the standard-setting 
procedure.) 


The teachers reviewed the multiple-choice 
parts of both 1985 and 1989 tests. They set 
scores that would represent the standards for 
each test: the acceptable standard and the 
standard of excellence. Table 7-2 shows the 
results of the standard-setting process. 


Table 7-2 
Grade 3 English Language Arts 
Comparison of Percentage of Students Achieving Standards 
1985 and 1989 


Total 1985 
1989 
Part A: Writing | 1985 25 
1989 2 
Part B: Reading} 1985 36 
1989 40 


The results revealed that nearly the same 
number of students met the standards in 1985 
and in 1989. Since a change of one in the 
raw score representing the standard will 
result in a change of approximately four per 
cent in the percentage of students meeting 
the standards, the differences shown are not 
important. 


The standard-setting results showed that, in 
the judgment of the standard-setters, students 
achieved the acceptable standard and the 
standard of excellence as frequently in both 
test administrations. 


Percentage of Score 


Acceptable Standard 
T, Year Maximum Score 
Score Representing Students Achieving] Representing Students Achieving 
Standard 
Standard Standard 


At or Above Standard 


Standard of Excellence 


Percentage of 


At or Above 


CONCLUSION 


The results of the test-equating analysis 
(shown in Table 7-1) indicate that achieve- 
ment in those aspects of language arts that 
were tested was higher in 1989 than in 1985, 
but similar percentages of students achieved 
acceptable or excellent standards (shown in 
Table 7-2). Because the test-equating pro- 
cess yields more precise results than the 
standard-setting process, it was concluded 
that achievement in language arts was 
somewhat higher in 1989 than in 1985. 
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GRADE 6 SOCIAL STUDIES 


In 1989, the Grade 6 Social Studies COMPARISON THROUGH 
Achievement Test was administered to RE-ADMINISTERING THE 1985 TEST 
29 995 students enrolled in the Grade 6 
Social Studies program. Ten schools in 10 jurisdictions were selected 
for the study. These schools were chosen to 
The 1985 and the 1989 Grade 6 Social be representative of rural, urban, large, and 
Studies achievement tests each included 50 small schools throughout the province. In 
multiple-choice questions. The 1985 test had June 1989, 244 Grade 6 students who were 
five written-response questions worth 30 writing the 1989 test also wrote the 1985 test. 
marks, and the 1989 test had seven written- Based on the results of the test-equating 
response questions worth 30 marks. process, all total scores for the students who 
wrote tests in 1989 were converted to esti- 
The average score in 1985 was 59.1%; the mated equivalent 1985 scores, and the 1985 
average score in 1989 was 62.5%. Grade 6 results and the 1989 Grade 6 results 
were then compared. (For an explanation of 
The special comparison study involved ad- the test-equating process, refer to Appen- 
ministering the 1985 test to a sample of dix D, page 64, and Appendix E, page 65.) 
students who were also writing the 1989 test Table 7-3 shows the results of the comparison. 


and using a single group of judges to deter- 
mine the standards for both tests. 


Table 7-3 
Grade 6 Social Studies 
Comparison of 1985 and 1989 Test Results 


Number of Marks 
Number of Students Writing 


Average Score after Equating 
Standard Deviation after Equating 
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The average 1989 score was higher than the 
average 1985 score by 0.2 marks out of a 
possible 100. A two-tailed t-test showed that 
this difference was statistically significant 
beyond the 0.05 level of probability. 
However, the reliability of the estimated 
scores is limited. Considering this limitation, 
the difference does not appear to reflect a 
real difference in achievement scores. 


STANDARD SETTING 


Twenty-one experienced Grade 6 Social 
Studies teachers from schools throughout the 


province were selected to participate in the 
standard-setting section of the study. (See 
Appendix C, page 63, for an explanation of 
the standard-setting procedure.) 


The teachers reviewed both 1985 and 1989 
tests. They set scores that would represent the 
standards for each test: the acceptable 
standard and the standard of excellence. 

Table 7-4 shows the results of the 
standard-setting process. 


Table 7-4 
Grade 6 Social Studies 
Comparison of Percentage of Students Achieving Standards 
1985 and 1989 


Acceptable Standard 


Year Maximum Score 
Test Score |Representing Students Achieving] Representing Students Achieving 
Standard At or Above Standard At or Above 
Standard Standard 


Total 1985 100 47 
1989 100 47 

Part A: 1985 50 22 
Multiple Choice | 1989 50 23 
Part B: 1985 30 15 
ritten Response} 1989 30 15 


The results revealed that more students met 
the standards in 1989 than in 1985. Since a 
change of one in the raw score representing 
the standard will result in a change of ap- 
proximately four per cent in the percent- 
age of students meeting the standard, the 
differences shown are not important. 


The standard-setting results showed that, in 
the judgment of the standard-setters, students 
achieved the acceptable standard and the 
standard of excellence as frequently in both 
test administrations. 


Percentage of Score 


Standard of Excellence 


Percentage of 


CONCLUSION 


The results of the test-equating analysis 
(shown in Table 7-3) indicate that achieve- 
ment in those aspects of social studies that 
were tested was about the same in 1985 and in 
1989, and similar percentages of students 
achieved the acceptable standards (shown in 
Table 7-4). Higher percentages of students 
met the standard of excellence in 1989 than in 
1985. Because the test-equating process 
yields more precise results than the standard- 
setting process, it was concluded that 
achievement in social studies was essentially 
the same in 1989 as in 1985. 
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GRADE 9 SCIENCE 


In 1989, the Grade 9 Science Achievement 
Test was administered to 27 201 students 
enrolled in the Grade 9 Science program. 


COMPARISON THROUGH 
RE-ADMINISTERING THE 1985 TEST 


Eight schools in eight jurisdictions were 


The achievement tests in both 1985 and 1989 selected to participate in the study. 


consisted of 75 multiple-choice questions. 


The average score in 1985 was 66.2%; the 
average score in 1989 was 66.8%. 


The special comparison study involved ad- 
ministering the 1985 test to a sample of 249 


These schools were chosen to be repre- 
sentative of rural, urban, large, and small 
schools throughout the province. In June 
1989, 249 Grade 9 students who were 
writing the 1989 test also wrote the 1985 
test. Based on the results of the test- 
equating process, all multiple-choice 


students who were also writing the 1989 test, scores for the 1989 population were 


using a single group of judges to determine 
the standards for both tests, and comparing 
equivalent questions. 


Average Score after Equating 
Standard Deviation after Equating 


The average 1985 score was 0.7% higher 
than the average 1989 score. A two-tailed 
t-test showed that this difference was 
Statistically significant beyond the 0.001 


converted to estimated equivalent 1985 

. scores, and the 1985 and the 1989 popula- 
tions were then compared. (For an 
explanation of the test-equating process, 
refer to Appendix D, page 64, and 
Appendix E, page 65, of this report.) 
Table 7-5 shows the results of the 
comparison. 


Table 7-5 
Grade 9 Science 
Comparison of 1985 and 1989 Test Results 


Number of Questions 
Number of Students Writing 


level of probability. However, this dif- 
ference may not be educationally 
significant. 
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STANDARD SETTING These teachers reviewed both 1985 and 
1989 tests. They set scores that would 


Twenty-one experienced Grade 9 Science represent the standards for each test: the 
teachers from schools throughout the acceptable standard and the standard of 
province were selected to participate in the excellence. Table 7-6 shows the results 
standard-setting section of the study. (See of the standard-setting process. 


Appendix C, page 63, for an explanation of 
the standard-setting procedure.) 


Table 7-6 
Grade 9 Science 
Comparison of Percentage of Students Achieving Standards 
1985 and 1989 


Acceptable Standard Standard of Excellence 


Year Maximum Raw Score Percentage of Raw Score Percentage of 
Score Representing Students Achieving |Representing Students Achieving 
Standard At or Above Standard At or Above 
Standard Standard 


1985 75 
1989 75 


The percentage of students meeting the ac- 1. They tested the same specific curric- 
ceptable standard in 1989 was virtually the ulum objectives. 
same as in 1985. 

2. They required the same level of 


However, the percentage of students meeting reading skill. 
the standard of excellence was higher in 1989 
than in 1985. 3. The wording was similar in terms of 
complexity and the quantity of in- 
formation presented. 
EQUIVALENT QUESTIONS 
4. They required the same number of 
Seven questions from the 1989 achievement steps or thought processes. 
test were judged to be equivalent to seven 
questions on the 1985 achievement test by 5. All three distractors were similar in 
the Grade 9 Science Test Review Committee terms of the process required to ar- 
and by the 21 teachers who participated in rive at those answers. 


the standard-setting process. 

The average score on the seven questions 
Questions were judged to be equivalent if was 65.8% in 1985 and 67.2% in 1989. 
they had all of the following characteristics: 
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CONCLUSION 


The results obtained from equating the 1985 
and the 1989 test results indicated that 
overall achievement in 1989 was virtually 
unchanged compared to 1985; however, 
standard-setting results indicated that a 
higher percentage of students achieved the 


standard of excellence in 1989 than in 1985. 
In addition, student achievement on the 
seven equivalent questions was slightly 
higher in 1989 than in 1985. 


It can be concluded that student achievement 
in Grade 9 Science in 1989 was the same as 
in 1985. 
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APPENDIX A 


USING ACHIEVEMENT TEST RESULTS 


A SYSTEMATIC APPROACH FOR THE 
EFFECTIVE USE OF ACHIEVEMENT TEST RESULTS 


Achievement test results can be used con- 
structively as one means of improving the 
quality of education. A systematic use of 
achievement test results would include the 
following steps: 


L. 


Comparing test results for a jurisdiction 
or school to the provincial results. Be 
sure that your comparisons include the 


*total test score, 

*total and subtest scores for multiple- 
choice questions, 

*total and subtest scores for written- 
response assignments (when 
appropriate), 

sindividual multiple-choice question 


results, and 
sindividual written-response 
question results (when appropriate). 


Noting any patterns, anomalies, and/or 
interrelationships in the results. 


Hypothesizing relationships between 
your observations and any of the factors 
listed in Section 2 of this report that 
may have had an effect on achievement 
or achievement test results. 


Considering and implementing a plan 
that will help to improve the quality of 
education for students. 


AN ADMINISTRATIVE MODEL FOR THE 
EFFECTIVE USE OF ACHIEVEMENT TEST RESULTS 


The following model may be useful for those 
who wish to develop a constructive system 
for interpreting achievement test results. 
This model is based on work done by Medi- 
cine Hat School District #76. 


BASIC PRINCIPLES 


E 


It is desirable and feasible for teachers 
and school administrators to make use 
of achievement test results in analysing 
the performance of their own students. 


It is more constructive for schools to 
develop their own analyses, interpret- 
ations, and action plans than to have 
these imposed externally. 


The impact of factors such as those 
listed in Section 2 should be analysed 
and discussed when reviewing achieve- 
ment test results. 


oat 


Subtest or reporting category results are 
usually more informative than total test 
scores. 


Generalizations should be stated with 
caution and should be supported by 
evidence that is independent of achieve- 
ment test results. 


It is neither desirable nor productive to 
compare the results of different schools. 


Achievement tests measure many of the 
objectives specified by the curriculum. 
However, skills and concepts that are 
not measured by the achievement tests 
are also to be taught and evaluated at 
the local level. 


Staff discussions as well as written 
reports are useful means of ensuring 
that results are appropriately interpreted 
and used. 


SUGGESTED CONTENT FOR 
INTERPRETATION OF INDIVIDUAL 
SCHOOL RESULTS 
1. Subject, grade level, and date of 
achievement test administration 
2. Number of students who wrote the 
achievement test 
3. Profiles of students or groups who 
wrote the achievement test, which 
include 
previous performances 
enumber of students repeating the 
grade, etc. 
4. Program emphases, such as hours of 
instruction, skill and content emphases 
5. Instructional practices, such as 
methodology, resources, and the 
relationship between the program 
offered and the provincial curriculum 
6. Program objectives not measured by 
the achievement test 
7. School results compared to provincial 
results on subtests 
8. Current school results compared to 
those of previous administration 
9. Discussion of item results, identi- 


fication of common student errors, and 
suggestions of ways for reducing the 
misunderstanding that leads to these 
errors 


10. Recommendations for the following 


year or semester 


11. Summary and concluding comments 


SUGGESTED REPORTING 
STRUCTURE 


Is 


Teachers and/or the principal analyse 
the results and prepare a written report 
about each administration of an 
achievement test. 


The principal reviews and signs the 
report. 


The report is shared with central office 
supervisory personnel. 


The appropriate central office super- 
visory personnel prepare a written 
response to the report, with copies of 
the response going to the teachers and 
the principal. 


If possible, all involved staff meet to 
discuss the report and the response. 


Reports are used to improve the pro- 


gram and maximize future opportunities 


for student success. 


When large differences exist between 
expected and actual achievement test 
results over time, consideration should 
be given to conducting a formal pro- 
gram evaluation. 
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APPENDIX B 


DEVELOPING ACHIEVEMENT TESTS 


The Student Evaluation and Records Branch 
develops achievement tests that measure 
student achievement at the grades 3, 6, and 9 
levels. Provincewide testing in Language 
Arts, Mathematics, Science, and Social 
Studies follows a four-year cycle for each 
grade level and subject. Many individuals 
and groups are involved in the development 
of each test: practising classroom teachers, 
school and central office administrators, and 
representatives of postsecondary institutions, 
the Curriculum Design Branch, the Lan- 
guage Services Branch, Regional Offices, 
and the Student Evaluation and Records 
Branch. Student Evaluation and Records 
Branch staff ensure the development of valid 
and reliable tests. 


Following is a summary of the phases of the 
test development process. 


1. Planning 

2. Approving Blueprints 

3. Developing Test Questions 
4 


. Constructing and Administering 
Field Tests 


. Analysing and Revising 

. Constructing Final Field Tests 

. Approving Final Field Tests 
Administering Final Field Tests 


co oN DA tw 


Constructing the Final Test 


10. Preparing and Administering the Final 
Test 


11. Marking 
12. Analysing and Reporting the Results 


Under normal circumstances it takes three 
years to complete all phases of the process. 


1. PLANNING 


Test developers ensure that the design of each 
achievement test reflects the goals and objec- 
tives of the Program of Studies and the 
curriculum specifications for each subject. 
Planning takes into consideration those parts 
of the program that are testable in a paper and 
pencil format, within a given time frame. 
Teachers and consultants from across the 
province assist in preparing the design of each 
test. 


Test developers prepare an interim test 
blueprint (an overall plan used to guide the 
development of a test). Questions that must 
be addressed at this point are: 


*What knowledge and skills should stu- 
dents be expected to possess? 


*What types of questions will constitute 
the test (multiple choice, short answer, or 
extended written response)? 


«What weighting will each part of the test 
be given? 


*How long and how demanding should the 
test be? 


*How should the results of the test be or- 
ganized for reporting purposes? 


In order to ensure that each test will produce 
meaningful and reliable results, test devel- 
opers incorporate statistical as well as cur- 
ricular standards in the test design. Statistical 
standards include projected test means, range 
of question difficulty, and requirements for 
reporting. For example, the ideal mean of a 
multiple-choice test containing questions with 
four alternatives is 62.5%. This is the 
midpoint between chance selection (25%) and 
perfection (100%). The range of difficulty of 
multiple-choice questions is expected to vary 
from 30% to 85% to ensure that students with 
varying ability levels are challenged. 
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Each dimension of the curriculum for which 
results are reported must contain at least six 
questions if the results are to be meaningful. 


2. APPROVING BLUEPRINTS 


Blueprint approval establishes the overall 
design of the test, the exact emphases given 
to each category for which results are 
reported, and the emphases given to the 
different cognitive levels. 


The interim blueprint is reviewed by a 
comunittee of Alberta Education personnel 
that represents the Curriculum Design 
Branch (or Language Services Branch), 
Regional Office consultants, and the Student 
Evaluation and Records Branch. This 
committee reviews the interim blueprint and 
makes recommendations to the Director of 
the Student Evaluation and Records Branch. 


The blueprint recommended by the Alberta 
Education committee is then reviewed by a 
Test Review Committee, which consists of 
members nominated by the Alberta 
Teachers’ Association, the Conference of 
Alberta School Superintendents, post- 
secondary institutions, and Alberta 
Education. This committee makes 
recommendations to the Director of the 
Student Evaluation and Records Branch. 


3. DEVELOPING TEST QUESTIONS 


Following blueprint approval, committees of 
practising classroom teachers working at the 
appropriate grade level are formed, and 
question development meetings are held. 
These committees develop new test ques- 
tions that reflect the goals and objectives of 
the Program of Studies and curriculum 
specifications. Where necessary, question 
developers are trained in the principles of 
question construction. Questions built in 
committee are then screened for format, 
validity, blueprint ‘fit’, and other design 
considerations. 


4. CONSTRUCTING AND 
ADMINISTERING FIELD TESTS 


After careful editing and formatting of 
questions developed by the teacher com- 
mittees, field tests are constructed. Any 
required artwork is completed during this 
phase of the test development process. 


With permission from school and jurisdiction 
personnel, field tests are sent to a number of 
teachers throughout Alberta. The students 
involved are representative of the student 
population for which the test has been de- 
signed. A minimum sample of 150 students 
writes each field test. 


Teachers who administer a field test are asked 
to comment in writing on the following: 


ereading level 


ehow closely the question matches the way in 
which a concept was taught 


elevel of difficulty of the questions 
equality of the questions and graphics 
eerrors of any kind 


The results from the administration of this 
initial round of field tests are used to validate 
content, to determine difficulty levels, and to 
ensure that questions are expressed clearly 
and unambiguously. 


5. ANALYSING AND REVISING 


The results of each field test are then analysed 
and scrutinized to determine whether indivi- 
dual questions require revision. Teacher 
comments regarding the way that test ques- 
tions are structured and the way that a subject 
is being taught are also carefully considered 
and used to guide revision. 


Questions deemed to require changes are re- 
vised and submitted for further field testing. 
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6. CONSTRUCTING FINAL FIELD TESTS 


Once the initial field test results are 
thoroughly analysed and questions requiring 
changes are revised, final field tests are 
constructed. These field tests follow the 
approved blueprint and parallel the actual 
achievement test in format and design. 


Final field tests, like all field tests, are 
submitted for further validity checking, 
editing, and proofreading. In grades 6 and 9, 
separate tests in English and in French are 
developed for language arts. At this point, 
all other tests for Grade 6 and Grade 9 are 
translated into French. 


7. APPROVING FINAL FIELD TESTS 


After the final field tests have been con- 
structed, a second meeting of the Alberta 
Education Committee that represents the 
Curriculum Design Branch (or Language 
Services Branch), Regional Office con- 
sultants, and the Student Evaluation and 
Records Branch is convened. This com- 
mittee reviews the final field tests and makes 
recommendations for improvement. 


The Test Review Committee, which 
approved the blueprint in Phase Two of the 
test development process, meets a second 
time to review and recommend for approval 
the final field tests and the instructions for 
administering the tests. If a test includes 
short-answer or extended-writing questions, 
the Test Review Commitee discusses 
standards of achievement and marking 
standards appropriate for the test. Again, 
this committee makes recommendations to 
the Director of the Student Evaluation and 
Records Branch. 


8. ADMINISTERING FINAL FIELD 
TESTS 


The final field tests are administered and the 
results are used as a final screen in selecting 
questions for placement on the provincial 


achievement test. A minimum sample of 250 
students writes each final field test. The 
sample is selected to include: 


eonly students who have received 
instruction in the course 


students representing a normal dis- 
tribution of ability levels 


estudents from rural and urban schools 


students from large and small schools 


9. CONSTRUCTING THE FINAL TEST 


The construction of the final test form is based 
upon information collected from the final field 
test administration. The Test Review Com- 
mittee is reconvened to review the final test 
form. eee : 


The test is submitted for final validity 
checking, editing, and proofreading. Grade 6 
and Grade 9 achievement tests, in subjects 
other than language arts, are translated into 
French at this time. 


For each test an information bulletin is pre- 
pared, outlining the design and nature of the 
upcoming tests. These bulletins are dis- 
tributed to each school in September to 
facilitate program and instructional planning 
by teachers and administrators. 


10. PREPARING AND ADMINISTERING 
THE FINAL TEST 


The completed achievement test is commer- 
cially printed and prepared for distribution. 
It is administered to the students by their 
classroom teachers. 


Sufficient copies of the test are mailed to each 
school. Quantities are based on the number of 
students enrolled in the subject as reported to 
the Student Evaluation and Records Branch by 
school superintendents. 
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11. MARKING 


All written-response sections of the tests are 
marked by classroom teachers. These 
teachers, who are recommended by their 
superintendents, are currently teaching the 
course being evaluated, have taught the 
course for a minimum of two years, and hold 
a valid permanent Alberta Professional 
Teaching Certificate. Student Evaluation 
and Records Branch staff train and supervise 
the teachers during the marking sessions. All 
multiple-choice responses are machine 
scored. 


12. ANALYSING AND REPORTING THE 
RESULTS 


Once the test has been written, at least 20 
classroom teachers review the test question 
by question, to judge the appropriateness of 
the standard built into the test. These 
teachers identify a test score that reflects 
student performance at a standard of 


excellence and a test score that reflects 
student performance at an acceptable level, 
based on the requirements of the Program of 
Studies. The teacher assessments are then 
compared to the actual levels of student 
achievement on a provincial basis. These 
results are reviewed by the Test Review 
Committee, which reconvenes a final time. 
This committee reviews the results of the test 
in terms of the objectives of the Program of 
Studies being measured. 


A statistical report is prepared and distributed 
to superintendents, school principals, Alberta 
Education officials, and other Departments of 
Education. This report is also made available 
to the general public. In addition to the Pro- 
vincial Report, each school and jurisdiction 
receives a statistical summary for its 
respective student population. 


For further information, please refer to the 
respective Achievement Test Bulletins, or call 
the Associate Director, Achievement Test and 
Diagnostic Evaluation Program, at 427-2948. 
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APPENDIX C 


STANDARD SETTING 


RATIONALE 


The purpose of standard setting in the 
Achievement Testing Program is to answer 
the question of whether provincewide per- 
formance is satisfactory. To use standard 
setting in this way requires two distinct 
judgments. The first is to establish what 
percentage of students tested can be expected 
to achieve at least an acceptable level of skill 
and knowledge required to proceed to the 
next level in that subject, assuming there are 
adequate teaching and resources. The 
second is to establish the test score that 
represents that level. 


Satisfactory provincial performance can be 
said to occur when the percentage of students 
scoring at or above the established test score 
is equal to or greater than the expected per- 
centage. Two similar judgments must be 
made for any other standards required, such 
as the level of skill and knowledge that re- 
flects excellence. 


Standard-setters must have a shared concept 
of the skills and knowledge of the borderline 
students for each standard set. Experience 
has shown that it is reasonable to judge that 
the expected percentage of students who 
should achieve the acceptable level is 85% 
and the expected percentage who should 
achieve at the level of excellence is 15%. 


ESSENTIAL ELEMENTS OF THE 
PROCEDURE USED 


1. Standard-setters were selected who 
were familiar with both the curriculum 
and the characteristics of the students 
who wrote the test. 


2. The rationale for and the purpose of 
standard setting were explained. 


3. The characteristics of borderline 
students were discussed, with emphasis 
on those characteristics that affect 
responses to the achievement test in 
question, and a consensus was reached 
for each standard set. 


4. Standard-setters made and recorded 
judgments on a question-by-question 
basis for the acceptable skills standard. 


5. Standard-setters made and recorded 
global judgments for the test at both the 
acceptable level and the standard of 
excellence. 


6. The raw score derived from 
question-by-question judgments for 
each standard-setter was calculated. 
The standard-setters were informed of 
their individual standards and of the 
median for the standard-setters as a 


group. 


7. The standard-setters were presented 
with data on the actual distribution of 
scores and the actual response fre- 
quencies. 


8. The standard-setters were allowed to 
revise their judgments, but it was 
stressed that they need not consider the 
actual results. 


9. The revised judgments were used to 
determine the test scores representing 
each level. 
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APPENDIX D 


EQUATING MULTIPLE-CHOICE TEST SCORES 


Comparing achievement in two different 
groups requires some common measure. 
The Student Evaluation and Records Branch 
develops new achievement tests for each 
administration, reflecting changes in cur- 
ricular emphases and refinements in test 
design learned from earlier tests. Thus, 
scores from one administration of an 
achievement test cannot be directly com- 
pared with scores from another admin- 
istration. 


Various techniques are available to address 
this problem. In order to compare the 1989 
achievement test results with results for the 
same subjects in 1985, the branch chose a 
variation of test equating as one of the 
techniques to be used. 


Each 1985 test was administered in 1989 to a 
sample of students who were also writing the 
1989 achievement test in that subject. The 
1985 tests were administered either one 
week before or one week after the 1989 tests 
were written, using the same instructions that 
were used in 1985. These students had not 
been exposed to the 1985 tests prior to 
writing them in 1989. 


The 1985 tests that were re-administered 
were scored using the same keys that were 
used in 1985. Scores were matched by 
student name and school with their scores on 
the 1989 achievement tests. Students for 
whom only one score was available were 
removed from the sample. 


The two sets of scores in the sample were 
then assumed to represent the same range of 
achievement, as the same sample of students 
had produced both sets. Both tests measured 
achievement in the same curriculum. 


Thus, achievement at a particular level in one 
set of scores should be equivalent to achieve- 
ment at the same level in another set of 
scores. A score at the 20th percentile on the 
1989 test would be equivalent to a score at the 
20th percentile on the 1985 test, because the 
percentiles were based on exactly the same 
students. (A percentile score represents the 
percentage of the sample of population 
scoring at or below that particular score.) 


For example, a score of 40 on the 1989 

Grade 9 Science Achievement Test fell at the 
18.7th percentile on the sample group. On the 
1985 test, the 18.7th percentile came between 
a score of 39 (percentile rank of 17.9), and 40 
(percentile rank of 19.7) in the sample group. 
Thus the score of 40 on the 1989 test was 
assigned an equivalent score of 39.4 on the 
1985 test. 


This fact was used to calculate equivalent 
scores for the two tests. The 1985 test was 
used as the anchor test. Each 1985 score was 
converted to its percentile equivalent. Per- 
centile scores for the 1989 scores in the sam- 
ple were also calculated. A transformation 
table was then produced. 


The transformation table was used to convert 
all scores in the 1989 population to their 
equivalents in the 1985 scores. It was then 
possible to treat the estimated scores as 
equivalent to the actual scores of the 1985 
population and perform a comparison using 
the two-tailed t-test to determine whether 
there were statistically significant differences 
between the achievement scores of students 
who wrote the 1985 test in 1985 and students 
who wrote the 1989 test in 1989. 
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APPENDIX E 


EQUATING WRITTEN-RESPONSE TEST SCORES 


The inclusion of a written-response section 
in the Grade 3 English Language Arts 
Achievement Test and the Grade 6 Social 
Studies Achievement Test required mod- 
ification to the test-equating method 
described in Appendix D, page 64, for tests 
with multiple-choice sections only. A 
consideration was the possible difference in 
the application of the scoring system between 
1985 and 1989. Scale descriptors have been 
revised and marker training procedures have 
been further refined, so scores cannot be 
assumed to be comparable. It was necessary 
to develop procedures to overcome these 
discrepancies. 


PROCEDURES 


The 1985 written-response tests were ad- 
ministered to the same sample that wrote the 
1985 multiple-choice tests in 1989. The 
1985 language arts test and the categorically 
scored portion of the social studies test were 
then scored using the 1989 scale descriptors 
by markers who had participated in the 
scoring of the 1989 written-response tests. 
The 1985 scoring guides were used for the 
analytically scored portions of the social 
studies test (questions 1 to 4). Samples of 
about 250 papers written in 1985 were 
re-scored at the same time by the same 


markers. Markers did not know the year in 
which any particular paper had been written. 


ANALYSES 


Results of the 1989 scoring of the 1985 
papers written in 1985 were compared with 
results of the 1985 scoring for the same 
paper. Conversion tables were generated 
using the equipercentile method described in 
Appendix D. These tables were used to con- 
vert the written-response scores for the 1985 
papers written and scored in 1989 to the 
same scoring standard that was applied in 
1985. 


Total scores for the 1985 papers written in 
1989 were then calculated using the 
multiple-choice scores produced by using the 
same key that was used in 1985 and the 
adjusted written-response scores. These total 
scores were then compared with the same 
students’ total scores on the 1989 tests in 
order to produce conversion tables based on 
the entire test. This procedure, using entire 
test scores while still adjusting written- 
response scores for changes in marking 
standards, gives more reliable results than 
separate analyses for the two parts of the 
tests (written response and multiple choice). 
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APPENDIX F 


REPORTING TO PARENTS 
ANSWERS TO FREQUENTLY ASKED QUESTIONS 


What are the achievement tests? 


The achievement tests are provincial 
government tests administered in Alberta 
schools to grades 3, 6, and 9 students in 
language arts, social studies, science, and 
mathematics. 


What is the purpose of the achievement tests? 


The achievement tests provide information 
about what students know and can do in 
language arts, social studies, science, and 
mathematics. The tests enable Alberta 
Education to monitor the level of achieve- 
ment of students throughout Alberta. The 
results also help local school boards, prin- 
cipals, and teachers identify the strengths and 
weaknesses in their implementation of these 
subjects. 


How many achievement tests will my child 
have to write? 


Students write only one achievement test 

in Grade 3, one in Grade 6, and one in 

Grade 9. Tests are rotated so that a different 
subject is tested each year. In 1989, Grade 3 
students wrote the language arts achievement 
test, Grade 6 students wrote the social studies 
achievement test, and Grade 9 students wrote 
the science achievement test. In 1990, 
Grade 3 students will write the mathematics 
achievement test, Grade 6 students will write 
the science achievement test, and Grade 9 
students will write the language arts 
achievement test. 


How should I prepare my child to write an 
achievement test? 


No preparation beyond normal classroom 
instruction is required to write an achieve- 
ment test. While students should be en- 
couraged to do their best, a good night’s 
sleep and a relaxed, confident approach to 
testing are the best possible preparation. 


How much do these tests count for my child? 


The achievement tests do NOT affect 
students’ final marks. The classroom 
teacher is responsible for evaluating stu- 
dents and awarding final marks. Achieve- 
ment test results are not released by Alberta 
Education until October, long after 
students’ marks have been determined by 
the classroom teacher. 


How do achievement test results help 
classroom teachers? 


Achievement test results provide feedback 
on student achievement to school boards, 
principals, and teachers. For example, 
teachers in a school that consistently scores 
high on one part of the curriculum but low 
on another may wish to examine their 
programs to see if changes are needed to 
achieve a better instructional balance. 


What are the limitations of the achievement 
tests? 


Paper and pencil tests cannot easily measure 
such things as laboratory skills, small group 
discussions, and creative thinking. Thus, 
some student strengths can be evaluated 
only by the classroom teacher. Also, a 
single test cannot reveal as much about a 
student’s development and growth as can 
evaluation by the classroom teacher over 
the course of a full school year. 


What advantage do achievement tests have 
over other standardized tests? 


Unlike commercially developed tests, 
achievement tests are based specifically 

on Alberta’s programs of study and 

are designed, written, and evaluated by 
experienced classroom teachers from across 
the province. Tests developed elsewhere 
may not reflect curriculum or standards 
appropriate for Alberta. 
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How do I interpret achievement test results? 


The Achievement Testing Program 
Provincial Report includes guidelines for 
interpreting results. Readers are cautioned 
not to overgeneralize conclusions based on a 
single administration of the test. Results 
should be compared to expectations or with 
the results of previous achievement tests in 
the same subject. Any trends that are ob- 
served in the scores must then be inter- 
preted in the context of a variety of factors 
that could affect student achievement, such 
as the school and community environment, 
students’ socioeconomic background, and 
available learning resources. 


Comparisons between districts, schools, or 
classrooms are likely to prove misleading 
and are therefore discouraged. 


Can I find out how my child did on the 
achievement test? 


Individual results on the achievement tests 
are made available to school principals. 


Since the tests are designed to gather 
information on groups of students, not on 
individuals, individual results must be 
interpreted with caution. 


Where can I get additional information about 
the Achievement Testing Program? 


Bulletins describing the content of the coming 
year’s achievement tests and the Provincial 
Report describing the results of the previous 
year’s testing are distributed to schools each 
fall. Requests for copies of these publications 
or questions and comments regarding the 
Achievement Testing Program should be 
directed to: 


Mr. Dennis Belyk 

Associate Director 

Achievement Tests and Diagnostic Unit 
Student Evaluation and Records Branch 
Alberta Education 

Devonian Building, West Tower 

11160 Jasper Avenue 

Edmonton, Alberta T5K 0L2 
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APPENDIX G 


RESULTS IN RELATION TO STANDARDS 


The discussion in the main body of this 
report deals with results only for those 
students who wrote the June 1989 
achievement tests. Some students were 
exempt from the tests, and others were 
absent on the day of testing. This appendix 
presents another way of looking at the 
provincial results in relation to standards. 
The percentage meeting standard is affected 
by which students are included in the 
population. 


We are interested in the differences between 
the percentage of students meeting the 


acceptable standard when absentees and 
exempted students are included as part of 
the population compared to when they are 
not included. 


This appendix presents the results in 
relation to the acceptable standard for 
Grade 3 English Language Arts, Grade 6 
Social Studies, and Grade 9 Science. The 
acceptable standard was established by a 
standard-setting procedure (see Appendix 
C, page 63). 


GRADE 3 ENGLISH LANGUAGE ARTS 


In order to achieve the acceptable standard 
for the Grade 3 English Language Arts 
Achievement Test, students needed a 
combined score of 51 marks out of 100. 


This part of the report gives the percentage 
of students achieving and not achieving the 
acceptable standard. Grade 3 students in 
Francophone or French Immersion programs 
are not included in the analysis, neither 
among those writing nor among those 


Figure G-1 


exempt, although many of them did write 
the test. Because of the reduced English 
Language Arts instruction in the first three 
grades in these programs, the standards are 
inappropriate for these two groups. 


Figure G-1 presents the percentage of 
students achieving the acceptable standard 
and not achieving the acceptable standard 
based on those students who wrote the test 
and were included in the analysis. 


Grade 3 English Language Arts 


Percentage of Students Achieving the Acceptable Standard 
Based on the English Language Program Students Who Wrote the Test 


Achieved Standard 


June 1989 


in Did Not Achieve Standard 
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Figure G-2 shows the percentage of students those students who wrote the test and were 


achieving the acceptable standard and not included in the analysis and those who were 
achieving the acceptable standard based on absent or exempt. 
Figure G-2 


Grade 3 English Language Arts 


Percentage of Students Achieving the Acceptable Standard 
Based on All Students in the English Language Program 
June 1989 


Achieved Standard = Absent ee fe Exempt TM Did Not Achieve Standard 


GRADE 6 SOCIAL STUDIES 


In order to achieve the acceptable standard exempt, although many of them did write 
for the Grade 6 Social Studies Achievement the test in Franch translation, and a few 
Test, students needed a combined score of wrote in English. We realize that language 
47 marks out of 100. of testing and language of instruction are 


important factors in student achievement. 
This part of the report gives the percentage 


of students achieving and not achieving the Figure G-3 presents the percentage of 
acceptable standard. Grade 6 students in students achieving the acceptable standard 
Francophone or French Immersion programs and not achieving the acceptable standard 
are not included in the analysis, neither based on those students who wrote the test 
among those writing nor among those and were included in the analysis. 

Figure G-3 


Grade 6 Social Studies 


Percentage of Students Achieving the Acceptable Standard 
Based on the English Language Program Students Who Wrote the Test 
June 1989 


} 18.4% | 4% 


Achieved Standard i Did Not Achieve Standard 
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Figure G-4 shows the percentage of students those students who wrote the test and were 
achieving the acceptable standard and not included in the analysis and those who were 
achieving the acceptable standard based on absent or exempt. 

Figure G-4 


Grade 6 Social Studies 


Percentage of Students Achieving the Acceptable Standard 
Based on All Students in the English Language Program 
June 1989 


Achieved Standard  ] Absent Exempt [[]] Did Not Achieve Standard 


GRADE 9 SCIENCE 
In order to meet the acceptable standard for Francophone and French Immersion 
the Grade 9 Science Achievement Test, students were not included in this analysis. 
students had to achieve a raw score of We realize that language of instruction and 
38 marks out of 75. language of testing are important factors in 


student achievement. 
This part of the report gives the percentage 


of students achieving and not achieving the Figure G-5 presents the percentage of 
acceptable standard. These results are based students achieving the acceptable standard 
only on students who were enrolled in the and not achieving the acceptable standard 
English language program and not on the based on those students who wrote the test 
total population. Results achieved by and were included in the analysis. 

Figure G-5 


Grade 9 Science 
Percentage of Students Achieving the Acceptable Standard 
Based on the English Language Program Students Who Wrote the Test 
June 1989 
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Figure G-6 shows the percentage of students those students who wrote the test and were 


achieving the acceptable standard and not included in the analysis and those who were 
achieving the acceptable standard based on absent or exempt. 
Figure G-6 


Grade 9 Science 
Percentage of Students Achieving the Acceptable Standard 
Based on All Students in the English Language Program 
June 1989 


Achieved Standard = Absent 


ne 


ACHIEVEMENT TESTING PROGRAM 


PROVINCIAL REPORT 
QUESTIONNAIRE 

The Student Evaluation and Records Please take a moment to respond to the 
; Branch strives to produce documents that following questions. Then detach this sheet and 

will be useful to the educational com- send it to: 
' munity. The purpose of the following 
questionnaire is to garner your opinions Mr. Michael Robinson 
| about the Provincial Report so that these Assistant Director, Data Analysis and 
opinions can be considered when the Student Records Services 
content and format of the report are Student Evaluation and Records Branch 
reviewed prior to June 1990. Alberta Education 
i 11160 Jasper Avenue 
7 Edmonton, Alberta T5K 0L2 
| — : - = 
USE OF THE REPORT 
) 

1. Please check the box beside the statement that applies to you. 


My present role is primarily that of 


@ teacher [_] 
1 
¢ school administrator [ ] 


. 2 
| @ central office administrator [| 


3 
e school board member [| 


© other (please specify) 


2. Please check the box beside the statement that applies to you. 


I read the report, but I DID NOT use it to interpret the results attained by my students. [| 


I read the report, and I used it to interpret the results attained by the students in 


@ my classroom @ my school @ my jurisdiction 
| 6 q 8 


3. Please respond to the following statement if you have checked boxes 6, 7, or 8 above. 
I have made use of the results to alter the educational program offered in 


@ my classroom @ my school @ my jurisdiction 


CONTENT OF THE REPORT 


Please judge the usefulness of the information included in the various sections of the 
report by checking the appropriate boxes below. 


Very Adequate Of Some Of No 
Useful for Use Use Use 


Section 1: Summary of Achievement 
Test Results 

12 
Section 2: Guidelines for 
Interpreting Achievement Test 
Results 


Sections 3 to.5: Specific 
Achievement Test Results 


Section 6: Student Achievement 
by Gender 


Section 7: Student Achievement 
Over Time 


| | [| | | =| | 
| af | =| | | | =| | 


If you wish, please comment further on the content of the report in the space below. 


FORMAT OF THE REPORT 


Please judge the usefulness of the report's format by checking the appropriate boxes below. 


Very Adequate Of Some Of No L 
Useful for Use Use Use 


Organization into Separate 
Sections 


Double—Column Presentation 
of Text 


Presentation of Figures 
Presentation of Tables 


Blending of Information 
in Text, Figures, and Tables 


O00 0-0 
ooo 0-0 
0000-0 
o0000 


If you wish, please comment further on the format of the report in the space below. 


